M19 GC WITSML Parser: work-product-component--WellLog, work-product-component--WellboreMarkerSet records not created
When attempting to ingest a WITSML well log file, a work-product-component--WellLog record is not created. Here are the steps used to ingest a WITSML well log file: M19-GC-WITSML-Log-Michael.txt Here is the WITSML well log file:Log_witsml.xml
I am expecting a well log record to be created with the id: m19:work-product-component--WellLog:WELLLOGAUTOTEST_123_M19 However, when calling the storage API to fetch the m19:work-product-component--WellLog:WELLLOGAUTOTEST_123_M19 record a 404 error is returned
curl --location 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/storage/v2/records/m19:work-product-component--WellLog:WELLLOGAUTOTEST_123_M19' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: m19' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer ...'
Response:
{
"code": 404,
"reason": "Record not found",
"message": "The record 'm19:work-product-component--WellLog:WELLLOGAUTOTEST_123_M19' was not found"
}
When checking the airflow logs https://548a04f7b3aa4e52a897b94772864397-dot-us-central1.composer.googleusercontent.com/log?dag_id=Energistics_xml_ingest&task_id=witsml_parser_task&execution_date=2023-08-06T19%3A27%3A13.290211%2B00%3A00, a manifest is created that has a work-product-component--WellLog record with this id:
[2023-08-06, 19:29:44 UTC] {pod_launcher.py:291} INFO - Running command... kill -s SIGINT 1
[2023-08-06, 19:29:45 UTC] {pod_launcher.py:168} INFO - {"manifest": {"kind": "osdu:wks:Manifest:1.0.0", "ReferenceData": [], "MasterData": [], "Data": {"WorkProduct": {"acl": {"owners": ["data.default.owners@m19.group"], "viewers": ["data.default.viewers@m19.group"]}, "kind": "osdu:wks:work-product--WorkProduct:1.0.0", "legal": {"legaltags": ["m19-Test-Legal-Tag-9795887"], "otherRelevantDataCountries": ["US"], "status": "compliant"}, "createTime": "2018-01-23T08:18:18.731000-06:00", "createUser": "1411_input", "data": {"ResourceSecurityClassification": "m19:reference-data--ResourceSecurityClassification:RESTRICTED:", "AuthorIDs": [], "BusinessActivities": ["Drilling"], "Components": ["m19:work-product-component--WellLog:WELLLOGAUTOTEST_123_M19:"], "CreationDateTime": "2018-01-23T08:18:18.731000-06:00", "IsDiscoverable": true, "IsExtendedLoad": false, "Name": "aa3fb285caf9414f8cf67bd6e2c61a60", "ExtensionProperties": {}}, "id": "m19:work-product--WorkProduct:078b6aad-dec7-403b-b3ab-c599992ab9e5", "modifyTime": "2003-11-24T08:17:00+00:00", "version": 1}, "WorkProductComponents": [{"acl": {"owners": ["data.default.owners@m19.group"], "viewers": ["data.default.viewers@m19.group"]}, "kind": "osdu:wks:work-product-component--WellLog:1.0.0", "legal": {"legaltags": ["m19-Test-Legal-Tag-9795887"], "otherRelevantDataCountries": ["US"], "status": "compliant"}, "createTime": "2018-01-23T08:18:18.731000-06:00", "createUser": "1411_input", "data": {"ResourceSecurityClassification": "m19:reference-data--ResourceSecurityClassification:RESTRICTED:", "Datasets": ["m19:dataset--File.WITSML:WELLLOGAUTOTEST_123_M19:"], "IsDiscoverable": true, "IsExtendedLoad": false, "AuthorIDs": ["1411_input"], "CreationDateTime": "2018-01-23T08:18:18.731000-06:00", "Name": "L001", "BottomMeasuredDepth": 509.01, "Curves": [{"BaseDepth": 509.01, "CurveID": "21c7c490-6210-4406-9869-00a132640148", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.556894+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Mdepth", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "692df092-5079-42f2-8230-46ed2031dc7a", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.556967+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Vdepth", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "96721075-f63c-4638-b810-892824c043f1", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557011+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Bit Dist", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "7b004068-40b3-4e7e-b180-42d28a813856", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557051+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "TQ on btm", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "75f6da19-1d06-4dff-b80d-ad988456b12f", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557090+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "TQ off btm", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "940d5bab-4f6d-4dd0-94c4-0393b800dde4", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557127+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "ROP", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "1730bf53-c267-493e-99d3-64f181138c1c", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557173+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "WOP", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "a000d81a-c34b-4dea-84b2-d68e6511942d", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557213+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "HKLD", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "9b3f00d6-7a0f-4420-9c1c-f5703ce7dd76", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557250+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Surf RPM", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "0750047a-526c-444a-9cba-810223846c3b", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557287+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Mtr RPM", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "c5fbad4b-c698-4186-bb9f-01212fac6222", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557323+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Avg TQ", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "8ef3475d-c234-468d-94a7-9856a8caf740", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557362+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Max TQ", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "067f5a2b-ad48-405f-bf62-0d560afdf269", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557420+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Min TQ", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "3ad709b3-c59f-4fd2-b9ca-f71fa9eba661", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557459+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Max - Min TQ", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "f2533499-a0c0-4335-a282-c5a22d23b3bb", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557497+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Max - Min TQ", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "dca2b2e1-f56f-4211-8545-80c21b6487c8", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557562+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Pump p avg", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "6034b3a0-9c50-44fa-a210-6100b6874773", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557600+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Mud D avg", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "890d78eb-3a39-40e9-add8-15d8f3c0ffa5", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557639+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Mud Temp avg", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "499e6d11-55c9-4e97-a1ee-d09efd44bdc2", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557675+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "Bit RPM", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "876af848-4901-40bc-93e6-e9f4a9004306", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557770+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "DXC", "TopDepth": 499.0}, {"BaseDepth": 509.01, "CurveID": "dd2aaff8-1389-49a5-b5dc-02912c885ab8", "CurveUnit": "m19:reference-data--UnitOfMeasure:m:", "DateStamp": "2023-08-06T19:27:59.557871+00:00", "DepthUnit": "m19:reference-data--UnitOfMeasure:m:", "Interpolate": true, "IsProcessed": true, "Mnemonic": "ECD", "TopDepth": 499.0}], "LoggingDirection": "decreasing", "LogRun": "12", "ServiceCompanyID": "m19:master-data--Organisation:Dana:", "TopMeasuredDepth": 499.0, "WellboreID": "m19:master-data--Wellbore:WELLBOREAUTOTEST_123_M19:", "WellLogTypeID": "m19:reference-data--LogType:Raw:"}, "id": "m19:work-product-component--WellLog:WELLLOGAUTOTEST_123_M19", "modifyTime": "2003-11-24T08:17:00+00:00"}], "Datasets": [{"acl": {"owners": ["data.default.owners@m19.group"], "viewers": ["data.default.viewers@m19.group"]}, "kind": "osdu:wks:dataset--File.WITSML:1.0.0", "legal": {"legaltags": ["m19-Test-Legal-Tag-9795887"], "otherRelevantDataCountries": ["US"], "status": "compliant"}, "createTime": "2018-01-23T08:18:18.731000-06:00", "createUser": "1411_input", "data": {"DatasetProperties": {"FileSourceInfo": {"FileSize": "33973", "FileSource": "/10200b30-5231-45eb-a697-f984d5f1dc0b/aa3fb285caf9414f8cf67bd6e2c61a60"}}, "TotalSize": "33973", "ResourceSecurityClassification": "m19:reference-data--ResourceSecurityClassification:RESTRICTED:", "Name": "aa3fb285caf9414f8cf67bd6e2c61a60", "SchemaFormatTypeID": "m19:reference-data--SchemaFormatType:EnergisticsWITSML:", "WITSML_Version": "2.0"}, "id": "m19:dataset--File.WITSML:WELLLOGAUTOTEST_123_M19", "modifyTime": "2003-11-24T08:17:00+00:00", "version": 1}]}}}
[2023-08-06, 19:29:45 UTC] {pod_launcher.py:198} INFO - Event: witsml-parser-task.10a1eadd9f31440c9888de59092453c0 had an event of type Running