AWS M12: openvds_import Dag fails when trying to convert segy file to openvds using seismic ddms
I am trying to follow the segy to openvds workflow described in https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M12/AWS-M12/DDMS%20Seismic/R3M12_AWS_OSDU_OpenVDS_DataProcessFlow.docx.
I am using the following: Environment file: https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M12/AWS-M12/Environment/Preshipping_r3-m12_Postman_Env.json Postman collection: https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M11/AWS-M11/DDMS%20Seismic/AWS%20-%20OpenVDS.postman_collection.json
I was able to upload a segy file to sdutil at location sd://osdu/subproject1/ST0202R08_TIME_2.segy.
I ran the following command to trigger the OpenVDS Workflow:
curl --location --request POST 'https://r3m12.preshiptesting.osdu.aws/api/workflow/v1/workflow/openvds_import/workflowRun' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: osdu' \
--header 'Authorization: Bearer eyJraWQiOiJjTktVd3pBeVNnb3htamNtRWZnYUJ4VGw4NnRaMjU3ZVhTZ05PNUh2VVk4PSIsImFsZyI6IlJTMjU2In0.eyJzdWIiOiJiMzRiOTYzYy1iOTlhLTRhNjUtOTBmZS1iNjBhZDBjNTNhMWUiLCJpc3MiOiJodHRwczpcL1wvY29nbml0by1pZHAudXMtZWFzdC0xLmFtYXpvbmF3cy5jb21cL3VzLWVhc3QtMV9NVm5WWU5QV0giLCJ2ZXJzaW9uIjoyLCJjbGllbnRfaWQiOiI0ZTV1aWIwMHFiaTZoM3FsaTRvaDltZTU5ZyIsIm9yaWdpbl9qdGkiOiI2YmU3YWI1Zi00YmZkLTRmM2MtYTI4OC1hNTRiOWFjYzcyMzMiLCJldmVudF9pZCI6Ijg0NDg3Zjk3LTEwOGMtNDQwMi1hMGVhLTczNzk4MjJiNjk1YSIsInRva2VuX3VzZSI6ImFjY2VzcyIsInNjb3BlIjoib3BlbmlkIGVtYWlsIiwiYXV0aF90aW1lIjoxNjU2NTMyNTA3LCJleHAiOjE2NTY2MTk3MzAsImlhdCI6MTY1NjYxNzkzMCwianRpIjoiMWM4ZmY5NGItNGZiYS00ZTU2LWEwOGMtYzNjN2Y1M2U1MzhlIiwidXNlcm5hbWUiOiJhZG1pbkB0ZXN0aW5nLmNvbSJ9.K2I6DRDDp1wMqD6MjqK92ExywPBvsWNQvKqSmoGhjnFjIaqBeQIzVaSr8h8eYYQuB-Ajody1ql2B50-XauX7mSP59bQ00f4omMUJ9VTOz4ZGt9WPKKht-3suly7ajGBP_Ev7gQLy1KY6l8--kjYg8z2vcldE6PTOqyhOrkvi8RiEkNcva_iQab5wKNPMEV49nz120kUPmOi5ml6TLP3tIzlRiP8N-vuEt3dBxWN_358HvoDyMqC0S6zH-5RC81b6QS_b1JTMG-koiRQZ2RK2On1BH20kLIfUc-KnGz3WGwROUaCu8HDGGFD3zZdiX5HDXqkQRPIHBpAbfstu2y-ecA' \
--data-raw '{
"executionContext": {
"data-partition-id": "osdu",
"url_connection":"Region=us-east-1;AccessKeyId=ASIAYFSQIKIRSVVEDTWY;SecretKey=ptYzb7gNxXBsVFd/EFrtTk8OiQqvjV7xE1zhNyDE;SessionToken=FwoGZXIvYXdzEB0aDLk87aoijNRgptyYGSLxA+N4hCxJhED+BkqqOXXZ0hsajKWDXogvyW1VbmvnZ+he/H9mdGVK6N2JVeh7m97C0FEcVSUiUpk6gFWFzW1+dTX9XlFUOcSjCuHEuEkVSV+axKoZC3drLDQi30SDu4ODkwoud+9bu3r8tmKnl7O6Sh/uVP4n+ZLyz0hF6slS9ordk7gpGzk348CZF3+ipQlUru3Sx/pJD/DTZb+rIoMkXxMZLP3VXdt1nS60kuDlFg4QSp1k+OyerGpfTkSUeFvcNaHHzvMj/aObnvEy44nNoEaqUjlIHSXHPdupeVlryC7P4bwov8RC0xdTdumrR+ww/AsZ4DLGMyDibgAyXFO1Oc01IgiiyDTUkVxXdq4Jag500DwKCNDboHe6R/HLjSTWlgBI7+I1Tyj/6A1vxs8+UOxqU2vRO60hotTeHoq4RuwHhih6194o2M01aurStd3Bxxwbu+jqr2+0Oari+NzhroKnDiQu8YaGY6cU8XEMrNJw96wUOmaeiRXJ6WSxcBVa6Hkr8cmi0Er0g4Mab+Y0sqaAO64GMHQMQQMkMmYrIr2DcfjPQd+JDIS70n55FH8ruGSSdzkL57tnadXLRiK2zJ3thS+jzuFqbDny9CQEFDXWwYOxdeQD8J573Y4EIDPNe/8Yhh9CfxL3SyAMjd0v34CIKL3395UGMi2Xb3c8tf+f7jLprFYK8/Qeu36tzvn0+O4ZOxEXdV09snVKpJzCjpESrVQ8BGU=;Expiration=2022-06-30T20:38:37.000+00:00",
"input_connection":"Region=us-east-1;AccessKeyId=ASIAYFSQIKIRYAIFMW2X;SecretKey=Vg/InNE+isg8t0TcRTxBOSj0Pfc08+0ESEsRhL0G;SessionToken=FwoGZXIvYXdzEBwaDFlp6Hu11A+BZ2bwCyKZBJOOgZCKcOW1VpY+CWfx72fI56pasMR7vE1YxpvxuFroqKa+UfktUAoRfeqcEd/LJzO1gt5doMjxx9J+yF6rV5JW8DinVaPyE1qDLtjuJ/8f9DykKf4kM/4LAyHSWexqbi6KhCVbHyhz7/+wuMG+IInUt/Iwe6Get0Zt/U1wbuP0ts7f3AI3hh0hAkXJzKzKoHH4G0zmF7c8MQT7/3ZsFnB94ItG8Sv9Z975h4OVSyBSLQv//6dG8YtJ5dNPh4Fa6n0zBM//KbbaeEPnyAAdn9nQraDC7rexHxZRIgjsNFMthZ+oQMHGD/2V9kpSGdAiqq3rePil2bIHIWcGCn/9kM9mcp3eT42vuZBYyI5SnmSv12FOPYO25vZeWi69j9OODF9H023DwpuUSzTwKIVM4BjyKXEQAdrb67ofW3qYLCoTBzRDJQqZQigWin8dBCzediTv6Ox9jkMk7W/ZXK83tg8SpGtIE4fBN1dzfBlCs+FhRVgCW7X9CNdX7UJAQCmAlrahvMiuDWops+toyI4xjZRAwLuE5+JAc352GwpxmWmRLlZJNUc040+lfsbb1mLFR41Xhwq24rvubOW2hevhhSW0e5pLln5YKjjxEQAZD4qksEyafxpYEwq5zpieI9suU9aEn5PDsd0j5kFoIIlb8EiAonksX95jymSYcy+DZPHXF9G6n53BGpP3YJcO5CHWaoVW+r2tNZqcWijD0feVBjItdpS7D2oKR65d6bMhjfWWjCF19P1gN1regn1RJ+oIIsECqzYvlEPtnT5XYurX;Expiration=2022-06-30T19:17:39.000+00:00",
"segy_file":"sd://osdu/subproject1/ST0202R08_TIME_2.segy",
"url":"sd://osdu/subproject1/"
}
}'
The output from triggering the request was the following:
{
"workflowId": "osdu:openvds_import",
"runId": "eed345ac-311d-4293-a873-7b00b2e6c6c0",
"startTimeStamp": 1656618191505,
"status": "submitted",
"submittedBy": "admin@testing.com"
}
The dag failed during the openvds tasks, I have attempted to trigger the task multiple different times with different seismic files and the openvds_import dag fails everytime.
The logs for the failed dag can be found here: https://r3m12.preshiptesting.osdu.aws/api/airflow/log?dag_id=openvds_import&task_id=OPENVDS&execution_date=2022-06-30T19%3A43%3A47.042173%2B00%3A00