Not able to perform Azure M14 WITSML parser testing for all data types
Hello,
The issue occurs for all data types: well, wellbore, trajectory (known issue), well log, tubular, marker.
AIRFLOW_CTX_DAG_OWNER=energistics AIRFLOW_CTX_DAG_ID=Energistics_xml_ingest AIRFLOW_CTX_TASK_ID=provide_manifest_integrity_task AIRFLOW_CTX_EXECUTION_DATE=2022-10-27T21:09:11.842597+00:00 AIRFLOW_CTX_DAG_RUN_ID=34c8ade5-9445-4624-9772-2f35614f380c [2022-10-27, 21:09:46 UTC] {init.py:146} DEBUG - Preparing lineage inlets and outlets [2022-10-27, 21:09:46 UTC] {init.py:190} DEBUG - inlets: [], outlets: [] [2022-10-27, 21:09:46 UTC] {ensure_manifest_integrity.py:77} DEBUG - Manifest data: {'kind': 'opendes:wks:Manifest:1.0.0', 'ReferenceData': [], 'MasterData': [{'acl': {'owners': ['data.default.owners@opendes.contoso.com'], 'viewers': ['data.default.viewers@opendes.contoso.com']}, 'kind': 'opendes:wks:master-data--Well:1.0.0', 'legal': {'legaltags': ['opendes-WITSML-Legal-Tag-Test7853197'], 'otherRelevantDataCountries': ['US'], 'status': 'compliant'}, 'createTime': '2022-10-27T21:09:33.536495+00:00', 'data': {'ResourceSecurityClassification': 'opendes:reference-data--ResourceSecurityClassification:RESTRICTED:', 'SpatialLocation': {'AsIngestedCoordinates': {'features': [{'type': 'AnyCrsFeature', 'geometry': {'type': 'AnyCrsPoint', 'bbox': [0.0, 0.0, 0.0, 0.0], 'coordinates': [0.0, 0.0]}, 'properties': {}}], 'persistableReferenceCrs': 'opendes:reference-data--CoordinateReferenceSystem:23031:', 'type': 'AnyCrsFeatureCollection', 'CoordinateReferenceSystemID': 'opendes:reference-data--CoordinateReferenceSystem:23031:'}, 'Wgs84Coordinates': {'features': [{'type': 'Feature', 'geometry': {'type': 'Point', 'coordinates': [0.0, 0.0]}, 'properties': {}}], 'type': 'FeatureCollection'}}, 'FacilityID': 'Govt-Number', 'FacilityName': 'Company Legal Name', 'VerticalMeasurements': [{'VerticalMeasurementID': 'KB', 'VerticalMeasurement': 78.5, 'VerticalMeasurementDescription': 'Kelly Bushing', 'VerticalMeasurementPathID': 'opendes:reference-data--VerticalMeasurementPath:Elevation:', 'VerticalMeasurementTypeID': 'opendes:reference-data--VerticalMeasurementType:KB:', 'VerticalMeasurementUnitOfMeasureID': 'opendes:reference-data--UnitOfMeasure:ft:', 'VerticalReferenceID': 'SL'}, {'VerticalMeasurementID': 'SL', 'VerticalMeasurementDescription': 'Sea Level', 'VerticalMeasurementPathID': 'opendes:reference-data--VerticalMeasurementPath:Elevation:', 'VerticalMeasurementTypeID': 'opendes:reference-data--VerticalMeasurementType:MSL:'}]}, 'id': 'opendes:master-data--Well:WELLAUTOTEST_123', 'modifyTime': '2022-10-27T21:09:33.536495+00:00', 'version': 1}], 'Data': {'WorkProduct': {}, 'WorkProductComponents': [], 'Datasets': [{'acl': {'owners': ['data.default.owners@opendes.contoso.com'], 'viewers': ['data.default.viewers@opendes.contoso.com']}, 'kind': 'opendes:wks:dataset--File.WITSML:1.0.0', 'legal': {'legaltags': ['opendes-WITSML-Legal-Tag-Test7853197'], 'otherRelevantDataCountries': ['US'], 'status': 'compliant'}, 'createTime': '2018-01-23T08:19:21.668000-06:00', 'createUser': '1411_input', 'data': {'DatasetProperties': {'FileSourceInfo': {'FileSize': '4417', 'FileSource': '/osdu-user/1666904379748-2022-10-27-20-59-39-748/ef79d0e4091b4aa9bb5728b2ce592e21'}}, 'TotalSize': '4417', 'ResourceSecurityClassification': 'opendes:reference-data--ResourceSecurityClassification:RESTRICTED:', 'Name': 'ef79d0e4091b4aa9bb5728b2ce592e21', 'SchemaFormatTypeID': 'opendes:reference-data--SchemaFormatType:EnergisticsWITSML:', 'WITSML_Version': '2.0'}, 'id': 'opendes:dataset--File.WITSML:WELLAUTOTEST_123', 'modifyTime': '2001-05-31T08:15:00+00:00', 'version': 1}]}} [2022-10-27, 21:09:46 UTC] {search_record_ids.py:71} DEBUG - Search query "opendes:reference-data--CoordinateReferenceSystem:23031" OR "opendes:reference-data--VerticalMeasurementPath:Elevation" OR "opendes:reference-data--UnitOfMeasure:ft" OR "opendes:reference-data--VerticalMeasurementType:MSL" OR "opendes:reference-data--ResourceSecurityClassification:RESTRICTED" OR "opendes:reference-data--VerticalMeasurementType:KB" OR "opendes:reference-data--SchemaFormatType:EnergisticsWITSML" [2022-10-27, 21:09:46 UTC] {connectionpool.py:228} DEBUG - Starting new HTTP connection (1): search.osdu-azure.svc.cluster.local:80 [2022-10-27, 21:09:49 UTC] {connectionpool.py:456} DEBUG - http://search.osdu-azure.svc.cluster.local:80 "POST /api/search/v2/query HTTP/1.1" 400 None [2022-10-27, 21:09:49 UTC] {authorization.py:137} ERROR - {"code":400,"reason":"Bad Request","message":"Failed to derive xcontent"} [2022-10-27, 21:09:49 UTC] {taskinstance.py:1718} ERROR - Task failed with exception Traceback (most recent call last): File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1334, in _run_raw_task
Request workflow trigger:
curl --location --request POST 'https://osdu-ship.msft-osdu-test.org/api/workflow/v1/workflow/Energistics_xml_ingest/workflowRun'
--header 'data-partition-id: opendes'
--header 'Authorization:
--header 'Content-Type: application/json'
--data-raw '{
"executionContext": {
"Payload": {
"AppKey": "test-app",
"data-partition-id": "opendes"
},
"Context": {
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"legal": {
"legaltags": [
"opendes-WITSML-Legal-Tag-Test7853197"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"kind": "::dataset--File.Generic:1.0.0",
"record_id": "opendes:dataset--File.Generic:6547d3f2-579b-453c-9d13-1fb37637b8a7",
"version": 5
}
}
}'
Response: { "workflowId": "Energistics_xml_ingest", "runId": "34c8ade5-9445-4624-9772-2f35614f380c", "startTimeStamp": 1666904951304, "status": "submitted", "submittedBy": "preshipping@azureglobal1.onmicrosoft.com" }