Open ZGY issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues2021-10-20T15:51:13Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/21SEGY to ZGY conversion on GCP platform fails. [GONRG-3630]2021-10-20T15:51:13ZKamlesh TodaiSEGY to ZGY conversion on GCP platform fails. [GONRG-3630]The postman collection that executes successfully on AWS, IBM fails on GCP.
Using the sdutil, the file was uploaded or copied to the created subproject for the test.
Following documentation was used to follow the steps
[text](https://com...The postman collection that executes successfully on AWS, IBM fails on GCP.
Using the sdutil, the file was uploaded or copied to the created subproject for the test.
Following documentation was used to follow the steps
[text](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/blob/master/doc/testing.md#test-data)
D:\OSDU\dev\seismic-store-sdutil-master>python -m venv sdutilenv
D:\OSDU\dev\seismic-store-sdutil-master>sdutilenv\Scripts\activate.bat
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil config init
[1] google
Select the cloud provider: 1
Insert the google (defaultEnv) application key:
sdutil successfully configured to use google (defaultEnv)
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>set ID_TOKEN="Get this from postman by executing the refresh_token request"
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod --idtoken=%ID_TOKEN%
autotesting
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil stat sd://odesprod/autotesting --idtoken=%ID_TOKEN%
- Uri: sd://odesprod/autotesting
- Legal Tag: odesprod-Wellbore-Legal-Tag-Test3816976
- Storage Class: REGIONAL
- Storage Location: US-CENTRAL1
- Access Policy: uniform
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod/autotesting --idtoken=%ID_TOKEN%
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil cp data/ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy sd://odesprod/autotesting/ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy --idtoken=%ID_TOKEN%
- Initializing transfer session ... OK
- Initializing resumable-transfer location ... OK
- Uploading Data [ 100% |██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 0.98G/0.98G - 12:23|00:00 - 1.41MB/s ]
- Transfer completed: 1.347307686813347 [MB/s]
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod/autotesting --idtoken=%ID_TOKEN%
ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy
The attached postman collection was used to ingest and convert the segy file to zgy.
A fully manual method (Way 1) is used in the collection where the following records are created FileCollection.SEGY, SeismicBinGrid, SeismicTraceData, and work-product. The document has to workflow service v1 & v2. Tried both
When version v2 was used got the message of endpoint/service was not found.
Even the doc the example for workflow service v2 shows it using v1 (`https://{path}/api/workflow/**v1**/workflow/{workflow-id}`
The airflow log does not give more detail other than ERROR - Task failed with exception.
This is the body content of the trigger request
{
"executionContext": {
"sd_svc_api_key": "",
"storage_svc_api_key": "",
"filecollection_segy_id": "{{file-collection-id}}",
"work_product_id": "{{work-product-id}}",
"data_partition_id": "{{data-partition-id}}",
"id_token": "{{id_token}}"
}
}
where various ids are saved from the manual steps executed prior to executing the trigger of the workflow request.
Attached are the postman collection used and the airflow log [AirflowLogOfFailedzgyComversion.txt](/uploads/befded524f11a4c42ece9fee0611f742/AirflowLogOfFailedzgyComversion.txt)
[SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json](/uploads/3bfd15a13e09d78eb8a23f93cc4036fa/SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json)
@ChrisZhangM9 - Release 0.12Aliaksandr Ramanovich (EPAM)Yan Sushchynski (EPAM)Aliaksandr Ramanovich (EPAM)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/19SEGY_TO_ZGY DAG does not return the correct status of the DAG when getting ...2021-10-19T18:44:31ZKamlesh TodaiSEGY_TO_ZGY DAG does not return the correct status of the DAG when getting the details for a specific workflow runwhen using the GET /v1/workflow/{workflow_name}/workflowRun/{runId} the SEGY_TOZGY DAG does not return the correct status.
It always returns the details with status = "submitted"even when the DAG has finished the execution. Which can be ...when using the GET /v1/workflow/{workflow_name}/workflowRun/{runId} the SEGY_TOZGY DAG does not return the correct status.
It always returns the details with status = "submitted"even when the DAG has finished the execution. Which can be verified by looking at the Airflow dashboard for the DAG.M9 - Release 0.12GregGreghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/20SEGY to ZGY conversion on Azure platform fails.2021-10-19T18:35:02ZKamlesh TodaiSEGY to ZGY conversion on Azure platform fails.The postman collection that executes successfully on AWS, IBM fails on Azure. In the Airflow log, the following error messages are seen - while registering DAG RBAC: access denied and BOOTSTRAP -- ERROR.
There are multiple segy-to-zgy-co...The postman collection that executes successfully on AWS, IBM fails on Azure. In the Airflow log, the following error messages are seen - while registering DAG RBAC: access denied and BOOTSTRAP -- ERROR.
There are multiple segy-to-zgy-conversion DAGs, I tried all of them. The log that is attached is using segy-to-zgy-conversion-0.10.0
Using the sdutil, I verified that the segy file is uploaded.
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil config init
[1] azure
Select the cloud provider: 1
Insert the azure (azureEnv) application key:
sdutil successfully configured to use azure (azureEnv)
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil auth login
Successfully logged into Azure SDUTIL.
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://opendes/autotesting
ST10010ZC11_PZ_PSDM_KIRCH_FULL_D.MIG_FIN.POST_STACK.3D.JS-017536.segy
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>
I have attached the postman collection and the airflow log[azure_segyToZgyConversionAirflowLog.txt](/uploads/1217079cd3f2f8c727474e0211ae24e2/azure_segyToZgyConversionAirflowLog.txt)
[SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json](/uploads/8836c93e8be06d3e7e24e33b55e0b9a3/SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json)
The environment file can be gotten from Platform Validation project and the following variables need to be added
segyToZgyWorkflowName = segy-to-zgy-conversion-0.10.0
subProjectName = autotesting
segyFileName = ST10010ZC11_PZ_PSDM_KIRCH_FULL_D.MIG_FIN.POST_STACK.3D.JS-017536.segyM9 - Release 0.12MANISH KUMARMANISH KUMAR