IBM M10 Patch Witsml Tubular Dag: Same dataset file used
When running a postman request to ingest a tubular witsml file, no new dataset record is created. Below is an example request run the dag workflow:
curl --location --request POST 'https://osdu-cpd-osdu.odi-osdu-og-fa7661852f2ab29a6be32f560b2f5573-0000.us-south.containers.appdomain.cloud/osdu-workflow/api/workflow/v1/workflow/Energistics_xml_ingest/workflowRun' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: opendes' \
--header 'Authorization: Bearer ...' \
--data-raw '{
"executionContext": {
"Payload": {
"AppKey": "test-app",
"data-partition-id": "opendes"
},
"Context": {
"legal": {
"legaltags": [
"opendes-ManifestDag-Legal-Tag-151089"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"acl": {
"owners": [
"data.default.owners@opendes.ibm.com"
],
"viewers": [
"data.default.viewers@opendes.ibm.com"
]
},
"kind": "opendes:wks:dataset--File.Generic:1.0.0",
"file_name": "3eab439a4dca41e1b7f8b348a0f461ae",
"preload_file_path": "s3://oc-cpd-opendes-staging-bucket/3eab439a4dca41e1b7f8b348a0f461ae",
"version": 1
}
}
}
'
Response:
{
"workflowId": "8012a0f2ed35423fb182f0350d605884",
"runId": "d8078450-7217-476e-a734-5488b75cc3e4",
"startTimeStamp": 1648006504398,
"status": "submitted",
"submittedBy": "preshipteama@osdu.opengroup.org"
}
In the airflow logs The same file is listed for each run: opendes:dataset--File.WITSML:11f9d820-0ada-41a9-a6e5-4a0bc3f3edf8
This dataset record appears to be modified each time the dag is run (Looking at the modifyTime field)