Manifest Ingestion DAG issueshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues2024-01-22T15:10:15Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/112Osdu_Ingest - fails to detect issue with GeoJSON structure in Spatial block2024-01-22T15:10:15ZDebasis ChatterjeeOsdu_Ingest - fails to detect issue with GeoJSON structure in Spatial blockEnvironment M22/Azure/Preship
Osdu_Ingest run ID = aaf9eaa0-beec-421f-8cbe-f87f6373ed4e
['opendes:master-data--SeismicAcquisitionSurvey:DC19JAN']
Input JSON payload has some issue with GeoJSON part.
validate_manifest_schema_task show...Environment M22/Azure/Preship
Osdu_Ingest run ID = aaf9eaa0-beec-421f-8cbe-f87f6373ed4e
['opendes:master-data--SeismicAcquisitionSurvey:DC19JAN']
Input JSON payload has some issue with GeoJSON part.
validate_manifest_schema_task shows clean log.
Later the record gets created.
When we check record using Storage, it shows Spatial data (WGS84 coordinates).
Search does not show that information any more.
Troubleshooting reveals that Spatial data is not indexed since GeoJSON syntax is incorrect.
"geo-json shape parsing error: must be a valid FeatureCollection attribute: SpatialLocation.Wgs84Coordinates",
The question - why did "schema validation" step not detect the problem and stop record creation?
cc @Yan_Sushchynski
See enclosed file with additional information.
[2024-01-19-EDS-Seismic.txt](/uploads/4175777071cda54bd465b41eb0264f30/2024-01-19-EDS-Seismic.txt)