IBM M22 - Unit Convervion - Problems when the Master Data is Triggered

Hello,

I was trying to complete my task FoR Unit Conversion for Manifest Ingest and got an problem when I triggered the Master Data. The ID expected at Airflow is not showing up. Here is what I've done:

1. Create Legal Tag

2. Ingestion Master and Refedence Data

POST {{WORKFLOW_HOST}}/workflow/Osdu_ingest/workflowRun

Body Curl Version

curl --location 'https://cpd-osdu.apps.osdu-preship.ibmodi.com/osdu-workflow/api/workflow/v1/workflow/Osdu_ingest/workflowRun' \
--header 'data-partition-id: opendes' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <Token>' \
--header 'Cookie: 5e7c4a992d2558462d1c9fcde6f1cee7=4ec498d778b559cf74a987cfbc06b21b' \
--data-raw '{
    "runId": "3bd48b11-e720-4284-892b-1a632c61799e",
    "executionContext": {
        "Payload": {
            "AppKey": "test-app",
            "data-partition-id": "opendes"
        },
        "manifest": {
            "kind": "osdu:wks:Manifest:1.0.0",
    "ReferenceData": [
        {
            "id": "opendes:reference-data--ResourceSecurityClassification:RESTRICTED",
            "kind": "osdu:wks:reference-data--ResourceSecurityClassification:1.0.0",
            "data": {
                "Source": "Volve Loading",
                "Name": "RESTRICTED"
            },
            "acl": {
                "owners": [
                "data.default.owners@opendes.ibm.com"
            ],
                "viewers": [
                "data.default.viewers@opendes.ibm.com"
            ]
            },
    	    "legal": {
            	"legaltags": [
                	"opendes-ManifestDag-Legal-Tag-3669675"
            ],
            	"otherRelevantDataCountries": [
                	"US"
            ],
            	"status": "compliant"
            }
        }
    ],
    "MasterData": [
        {
            "id": "opendes:master-data--SeismicAcquisitionSurvey:ST0202R08-233296",
            "kind": "osdu:wks:master-data--SeismicAcquisitionSurvey:1.0.0",
            "meta": [
                {
                    "kind": "Unit",
                    "name": "ms",
                    "persistableReference": "{\"abcd\":{\"a\":0.0,\"b\":0.001,\"c\":1.0,\"d\":0.0},\"symbol\":\"ms\",\"baseMeasurement\":{\"ancestry\":\"T\",\"type\":\"UM\"},\"type\":\"UAD\"}",
                    "unitOfMeasureID": "opendes:reference-data--UnitOfMeasure:ms:",
                    "propertyNames": [
                        "RecordLength"
                    ]
                },
                {
                    "kind": "Unit",
                    "name": "ft",
                    "persistableReference": "{\"abcd\":{\"a\":0.0,\"b\":0.3048,\"c\":1.0,\"d\":0.0},\"symbol\":\"ft\",\"baseMeasurement\":{\"ancestry\":\"L\",\"type\":\"UM\"},\"type\":\"UAD\"}",
                    "unitOfMeasureID": "opendes:reference-data--UnitOfMeasure:ft:",
                    "propertyNames": [
                        "ShotpointIncrementDistance",
                        "CableLength",
                        "CableSpacingDistance",
                        "MaxOffsetDistance"
                    ]
                }
            ],
            "data": {
                "Purpose": "Acquisition for Volve",
                "SeismicGeometryTypeID": "opendes:reference-data--SeismicGeometryType:3D:",
                "OperatingEnvironmentID": "opendes:reference-data--OperatingEnvironment:Offshore:",
                "ShotpointIncrementDistance": 25.0,
                "EnergySourceTypeID": "opendes:reference-data--SeismicEnergySourceType:Airgun:",
                "SourceArrayCount": 2,
                "SourceArraySeparationDistance": 100.0,
                "SampleInterval": 4.0,
                "RecordLength": 10200,
                "CableCount": 4,
                "CableLength": 6000.0,
                "CableSpacingDistance": 100.0,
                "MaxOffsetDistance": 6000.0,
                "FoldCount": 120,
                "VesselNames": [
                    "GECO ANGLER"
                ]
            },
            "acl": {
                "owners": [
                "data.default.owners@opendes.ibm.com"
            ],
                "viewers": [
                "data.default.viewers@opendes.ibm.com"
            ]
            },
    	    "legal": {
            	"legaltags": [
                	"opendes-ManifestDag-Legal-Tag-3669675"
            ],
            	"otherRelevantDataCountries": [
                	"US"
            ],
            	"status": "compliant"
        }
        }
    ]
    }
    }
}'

Response

{
    "workflowId": "9bc5ee62ca5c498db13453ec36b2922e",
    "runId": "9c748348-0c82-404f-b5ab-178892b37d6c",
    "startTimeStamp": 1705468720080,
    "status": "submitted",
    "submittedBy": "osdu-bvt@osdu.opengroup.org"
}

3. Workflow - Get Workflow Status

GET {{WORKFLOW_HOST}}/workflow/Osdu_ingest/workflowRun/9c748348-0c82-404f-b5ab-178892b37d6c

Body Curl Version

curl --location 'https://cpd-osdu.apps.osdu-preship.ibmodi.com/osdu-workflow/api/workflow/v1/workflow/Osdu_ingest/workflowRun/9c748348-0c82-404f-b5ab-178892b37d6c' \
--header 'Data-Partition-Id: opendes' \
--header 'Authorization: Bearer <Token>' \
--header 'Cookie: 5e7c4a992d2558462d1c9fcde6f1cee7=4ec498d778b559cf74a987cfbc06b21b' \
--data ''

Response

{
    "runId": "9c748348-0c82-404f-b5ab-178892b37d6c",
    "startTimeStamp": 1705468720080,
    "endTimeStamp": 1705468734017,
    "status": "finished",
    "submittedBy": "osdu-bvt@osdu.opengroup.org"
}

4.Xcom Summary

  • Access Airflow
  • Access DAG “Osdu_ingest”
  • Change the view from “Tree” to “Graph”
  • Check the runID the you got on postman
  • Click on the graph on “Process_single_manifest_file_task”
  • A window will pop up, click on “log” on the top
  • Change from “log” to “XCom"
record_ids	['opendes:reference-data--ResourceSecurityClassification:RESTRICTED']
skipped_ids	[]

What can be causing this?

Regards,

Juliana Fernandes