add support for feature-collection indexing

Data Definition team has harmonized the geo-shape payload on bulk-records. Any records that has geo-shape, must be GeoJson and must conform to the schema mentioned here. All shapes must be put in FeatureCollection.

Earlier implementation of indexer service was missing this key shape hence geo-shape indexing and search workflows was broken.

This MR adds support for FeatureCollection indexing. It also adds official opendes:wks:master-data--Wellbore:1.0.0 as integration test to validate index and search workflow.

This MR also fixes few schema parsing errors which we discovered while enabling opendes:wks:master-data--Wellbore:1.0.0 indexing.

testing/indexer-test-core/src/main/resources/features/indexrecord/indexRecord-schema-service.feature

@@ -3,11 +3,11 @@ Feature: Indexing of the documents
 
   Background:
     Given the schema is created with the following kind
-      | kind                                           | index                                          | schemaFile      |
-      | tenant1:indexer-int-test:sample-schema-1:1.0.4 | tenant1-indexer-int-test:sample-schema-1-1.0.4 | index_records_1 |
-      | tenant1:indexer-int-test:sample-schema-2:1.0.4 | tenant1-indexer-int-test:sample-schema-2-1.0.4 | index_records_2 |
-      | tenant1:indexer-int-test:sample-schema-3:1.0.4 | tenant1-indexer-int-test:sample-schema-3-1.0.4 | index_records_3 |
-      | tenant1:indexer-int-test:sample-schema-1:1.0.5 | tenant1-indexer-int-test:sample-schema-1-1.0.5 | index_records_1 |
+      | kind                                         | index                                        | schemaFile                 |
+      | tenant1:indexer:test-data--Integration:1.0.0 | tenant1-indexer-test-data--integration-1.0.0 | index_records_1            |
+      | tenant1:indexer:test-data--Integration:2.0.0 | tenant1-indexer-test-data--integration-2.0.0 | index_records_2            |
+      | tenant1:indexer:test-data--Integration:3.0.0 | tenant1-indexer-test-data--integration-3.0.0 | index_records_3            |
+      | tenant1:wks:master-data--Wellbore:2.0.3      | tenant1-wks-master-data--wellbore-2.0.3      | r3-index_record_wks_master |
 
   Scenario Outline: Ingest the record and Index in the Elastic Search
     When I ingest records with the <recordFile> with <acl> for a given <kind>
@@ -15,23 +15,30 @@ Feature: Indexing of the documents
     Then I should get the elastic <mapping> for the <type> and <index> in the Elastic Search
 
     Examples:
-      | kind                                             | recordFile               | number | index                                            | type              | acl                            | mapping                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
-      | "tenant1:indexer-int-test:sample-schema-1:1.0.4" | "index_records_schema_1" | 5      | "tenant1-indexer-int-test-sample-schema-1-1.0.4" | "sample-schema-1" | "data.default.viewers@tenant1" | "{"mappings":{"well":{"dynamic":"false","properties":{"acl":{"properties":{"owners":{"type":"keyword"},"viewers":{"type":"keyword"}}},"ancestry":{"properties":{"parents":{"type":"keyword"}}},"data":{"properties":{"Basin":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Country":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"County":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"EmptyAttribute":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Established":{"type":"date"},"Field":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Location":{"type":"geo_point"},"OriginalOperator":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Rank":{"type":"integer"},"Score":{"type":"integer"},"State":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellName":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellStatus":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellType":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"DblArray":{"type":"double"}}},"id":{"type":"keyword"},"index":{"properties":{"lastUpdateTime":{"type":"date"},"statusCode":{"type":"integer"},"trace":{"type":"text"}}},"kind":{"type":"keyword"},"legal":{"properties":{"legaltags":{"type":"keyword"},"otherRelevantDataCountries":{"type":"keyword"},"status":{"type":"keyword"}}},"namespace":{"type":"keyword"},"type":{"type":"keyword"},"version":{"type":"long"},"x-acl":{"type":"keyword"}}}}}" |
-      | "tenant1:indexer-int-test:sample-schema-3:1.0.4" | "index_records_schema_1" | 5      | "tenant1-indexer-int-test-sample-schema-3-1.0.4" | "sample-schema-3" | "data.default.viewers@tenant1" | "{"mappings":{"well":{"dynamic":"false","properties":{"acl":{"properties":{"owners":{"type":"keyword"},"viewers":{"type":"keyword"}}},"ancestry":{"properties":{"parents":{"type":"keyword"}}},"data":{"properties":{"Basin":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Country":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"County":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"EmptyAttribute":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Established":{"type":"date"},"Field":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Location":{"type":"geo_point"},"OriginalOperator":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Rank":{"type":"integer"},"Score":{"type":"integer"},"State":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellName":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellStatus":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellType":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"DblArray":{"type":"double"}}},"id":{"type":"keyword"},"index":{"properties":{"lastUpdateTime":{"type":"date"},"statusCode":{"type":"integer"},"trace":{"type":"text"}}},"kind":{"type":"keyword"},"legal":{"properties":{"legaltags":{"type":"keyword"},"otherRelevantDataCountries":{"type":"keyword"},"status":{"type":"keyword"}}},"namespace":{"type":"keyword"},"type":{"type":"keyword"},"version":{"type":"long"},"x-acl":{"type":"keyword"}}}}}" |
+      | kind                                           | recordFile               | number | index                                          | type                     | acl                            | mapping                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+      | "tenant1:indexer:test-data--Integration:1.0.0" | "index_records_schema_1" | 5      | "tenant1-indexer-test-data--integration-1.0.0" | "test-data--Integration" | "data.default.viewers@tenant1" | "{"mappings":{"well":{"dynamic":"false","properties":{"acl":{"properties":{"owners":{"type":"keyword"},"viewers":{"type":"keyword"}}},"ancestry":{"properties":{"parents":{"type":"keyword"}}},"data":{"properties":{"Basin":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Country":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"County":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"EmptyAttribute":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Established":{"type":"date"},"Field":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Location":{"type":"geo_point"},"OriginalOperator":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Rank":{"type":"integer"},"Score":{"type":"integer"},"State":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellName":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellStatus":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellType":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"DblArray":{"type":"double"}}},"id":{"type":"keyword"},"index":{"properties":{"lastUpdateTime":{"type":"date"},"statusCode":{"type":"integer"},"trace":{"type":"text"}}},"kind":{"type":"keyword"},"legal":{"properties":{"legaltags":{"type":"keyword"},"otherRelevantDataCountries":{"type":"keyword"},"status":{"type":"keyword"}}},"namespace":{"type":"keyword"},"type":{"type":"keyword"},"version":{"type":"long"},"x-acl":{"type":"keyword"}}}}}" |
+      | "tenant1:indexer:test-data--Integration:3.0.0" | "index_records_schema_3" | 7      | "tenant1-indexer-test-data--integration-3.0.0" | "test-data--Integration" | "data.default.viewers@tenant1" | "{"mappings":{"well":{"dynamic":"false","properties":{"acl":{"properties":{"owners":{"type":"keyword"},"viewers":{"type":"keyword"}}},"ancestry":{"properties":{"parents":{"type":"keyword"}}},"data":{"properties":{"Basin":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Country":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"County":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"EmptyAttribute":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Established":{"type":"date"},"Field":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Location":{"type":"geo_point"},"OriginalOperator":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"Rank":{"type":"integer"},"Score":{"type":"integer"},"State":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellName":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellStatus":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"WellType":{"type":"text","fields":{"keyword":{"type":"keyword","null_value":"null","ignore_above":256}}},"DblArray":{"type":"double"}}},"id":{"type":"keyword"},"index":{"properties":{"lastUpdateTime":{"type":"date"},"statusCode":{"type":"integer"},"trace":{"type":"text"}}},"kind":{"type":"keyword"},"legal":{"properties":{"legaltags":{"type":"keyword"},"otherRelevantDataCountries":{"type":"keyword"},"status":{"type":"keyword"}}},"namespace":{"type":"keyword"},"type":{"type":"keyword"},"version":{"type":"long"},"x-acl":{"type":"keyword"}}}}}" |
 
   Scenario Outline: Ingest the record and Index in the Elastic Search with bad attribute
     When I ingest records with the <recordFile> with <acl> for a given <kind>
     Then I should get the <number> documents for the <index> in the Elastic Search with out <skippedAttribute>
 
     Examples:
-      | kind                                             | recordFile        | number | index                                            | skippedAttribute | acl                            |
-      | "tenant1:indexer-int-test:sample-schema-2:1.0.4" | "index_records_2" | 4      | "tenant1-indexer-int-test-sample-schema-2-1.0.4" | "data.Location"  | "data.default.viewers@tenant1" |
-      | "tenant1:indexer-int-test:sample-schema-3:1.0.4" | "index_records_3" | 7      | "tenant1-indexer-int-test-sample-schema-3-1.0.4" | "data.GeoShape"  | "data.default.viewers@tenant1" |
+      | kind                                           | recordFile        | number | index                                          | skippedAttribute | acl                            |
+      | "tenant1:indexer:test-data--Integration:2.0.0" | "index_records_2" | 4      | "tenant1-indexer-test-data--integration-2.0.0" | "data.Location"  | "data.default.viewers@tenant1" |
 
   Scenario Outline: Ingest the record and Index in the Elastic Search with tags
     When I ingest records with the <recordFile> with <acl> for a given <kind>
     Then I should be able to search <number> record with index <index> by tag <tagKey> and value <tagValue>
 
     Examples:
-      | kind                                             | recordFile        | index                                            | acl                            | tagKey    | tagValue    | number |
-      | "tenant1:indexer-int-test:sample-schema-1:1.0.5" | "index_records_1" | "tenant1-indexer-int-test-sample-schema-1-1.0.5" | "data.default.viewers@tenant1" | "testtag" | "testvalue" | 5      |
+      | kind                                           | recordFile               | index                                          | acl                            | tagKey    | tagValue    | number |
+      | "tenant1:indexer:test-data--Integration:1.0.0" | "index_records_schema_1" | "tenant1-indexer-test-data--integration-1.0.0" | "data.default.viewers@tenant1" | "testtag" | "testvalue" | 5      |
+
+  Scenario Outline: Ingest the r3-record with geo-shape and Index in the Elastic Search
+    When I ingest records with the <recordFile> with <acl> for a given <kind>
+    Then I should be able search <number> documents for the <index> by bounding box query with points (<top_left_latitude>, <top_left_longitude>) and  (<bottom_right_latitude>, <bottom_right_longitude>) on field <field>
+
+    Examples:
+      | kind                                      | recordFile                   | number | index                                     | acl                            | field                                   | top_left_latitude | top_left_longitude | bottom_right_latitude | bottom_right_longitude |
+      | "tenant1:wks:master-data--Wellbore:2.0.3" | "r3-index_record_wks_master" | 1      | "tenant1-wks-master-data--wellbore-2.0.3" | "data.default.viewers@tenant1" | "data.SpatialLocation.Wgs84Coordinates" | 52                | -100               | 0                     | 100                    |