Skip to content
Snippets Groups Projects
Commit de98ccad authored by Eldor Pulatov [EPAM/GCP]'s avatar Eldor Pulatov [EPAM/GCP] Committed by Oleksandr Kosse (EPAM)
Browse files

GONRG-5890-Change-to-GC

parent 1dd9f8d2
No related branches found
No related tags found
1 merge request!99GONRG-5890-Change-to-GC
......@@ -5,7 +5,7 @@
* [DAG File Compilation](#dag-file-compilation)
* [DAG Release and Distribution](#dag-release-and-distribution)
* [DAG Deployment](#dag-deployment)
* * [GCP](#gcp)
* * [Google Cloud](#gc)
* [Requirements](#requirements)
* [Testing](#testing)
* * [Registering a Workflow](#registering-a-workflow)
......@@ -17,13 +17,13 @@
Airflow DAG for transformation from SEGY to OpenVDS
## GCP
## Google Cloud
The `dags/segy_to_vds_ssdms_conversion_dag.py` DAG stores output to Seismic DDMS.
The `dags/segy_to_vds_conversion_dag.py` DAG stores output to GCS bucket.
# DAG File Compilation
## GCP
## Google Cloud
The `dags/segy_to_vds_ssdms_conversion_dag.py` DAG file contains a number of placeholders `{| |}`, specifying to put proper values there, e.g.
```python
......@@ -38,12 +38,12 @@ To bootstrap the DAG file cloud providers provide their specific `bootstrap` [sc
# DAG Release and Distribution
## GCP
GCP DAG version is distributed as a generic package. During OSDU release the bootstrapped DAG file is published to the [Gitlab Generic Registy](https://community.opengroup.org/help/user/packages/generic_packages/index.md#publish-a-generic-package-by-using-cicd).
## Google Cloud
Google Cloud DAG version is distributed as a generic package. During OSDU release the bootstrapped DAG file is published to the [Gitlab Generic Registy](https://community.opengroup.org/help/user/packages/generic_packages/index.md#publish-a-generic-package-by-using-cicd).
# DAG Deployment
## GCP
For GCP, the bootstrapped DAG only uses Airflow variables to populate these values at runtime.
## Google Cloud
For Google Cloud, the bootstrapped DAG only uses Airflow variables to populate these values at runtime.
Before deploying this DAG, make sure the following Airlfow variables are set:
......
## GCP: DAG FILE RENDERING
## Google Cloud: DAG FILE RENDERING
```bash
# Install requirements
......@@ -10,7 +10,7 @@ export SEGY_VDS_DOCKER_IMAGE="<image_tag>"
export K8S_NODE_POOLS=<node_pools> # i.e. pool-1,pool-2 the value will be split by comma
export SD_SVC_URL=<ssdms_host> # i.e https://<ssdms_host>/seistore-svc/api/v3
OSDU_GCP_DEPLOYMENTS_SUBDIR="<path_to_folder>" # ie: path to GCP deployment scripts folder
OSDU_GCP_DEPLOYMENTS_SUBDIR="<path_to_folder>" # ie: path to Google Cloud deployment scripts folder
OSDU_GCP_DAG_DIRECTORY="<path_to_folder>" # ie: segy-to-vds-conversion/airflow/workflow-svc-v2
OUTPUT_FILE="<path_to_output_file>" # ie. to avoid original file overwriting
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment