SEGY to ZGY conversion on GCP platform fails. [GONRG-3630]
The postman collection that executes successfully on AWS, IBM fails on GCP. Using the sdutil, the file was uploaded or copied to the created subproject for the test. Following documentation was used to follow the steps text
D:\OSDU\dev\seismic-store-sdutil-master>python -m venv sdutilenv D:\OSDU\dev\seismic-store-sdutil-master>sdutilenv\Scripts\activate.bat (sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil config init
[1] google
Select the cloud provider: 1
Insert the google (defaultEnv) application key:
sdutil successfully configured to use google (defaultEnv)
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>set ID_TOKEN="Get this from postman by executing the refresh_token request"
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod --idtoken=%ID_TOKEN%
autotesting
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil stat sd://odesprod/autotesting --idtoken=%ID_TOKEN%
- Uri: sd://odesprod/autotesting
- Legal Tag: odesprod-Wellbore-Legal-Tag-Test3816976
- Storage Class: REGIONAL
- Storage Location: US-CENTRAL1
- Access Policy: uniform
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod/autotesting --idtoken=%ID_TOKEN%
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil cp data/ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy sd://odesprod/autotesting/ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy --idtoken=%ID_TOKEN%
- Initializing transfer session ... OK
- Initializing resumable-transfer location ... OK
- Uploading Data [ 100% |██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 0.98G/0.98G - 12:23|00:00 - 1.41MB/s ]
- Transfer completed: 1.347307686813347 [MB/s]
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod/autotesting --idtoken=%ID_TOKEN%
ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy
The attached postman collection was used to ingest and convert the segy file to zgy.
A fully manual method (Way 1) is used in the collection where the following records are created FileCollection.SEGY, SeismicBinGrid, SeismicTraceData, and work-product. The document has to workflow service v1 & v2. Tried both
When version v2 was used got the message of endpoint/service was not found.
Even the doc the example for workflow service v2 shows it using v1 (https://{path}/api/workflow/**v1**/workflow/{workflow-id}
The airflow log does not give more detail other than ERROR - Task failed with exception.
This is the body content of the trigger request
{
"executionContext": {
"sd_svc_api_key": "",
"storage_svc_api_key": "",
"filecollection_segy_id": "{{file-collection-id}}",
"work_product_id": "{{work-product-id}}",
"data_partition_id": "{{data-partition-id}}",
"id_token": "{{id_token}}"
}
}
where various ids are saved from the manual steps executed prior to executing the trigger of the workflow request.
Attached are the postman collection used and the airflow log AirflowLogOfFailedzgyComversion.txt
SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json