diff --git a/README.md b/README.md index 6eec16153a37b50a8b2c4d8c8dfca2793320d516..2b0b2b4d64ed2568171b8ebaae02fd6541552816 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,23 @@ Environment dependencies might be installed by several ways: 1. Setting up an environment into the Cloud Composer Console. 2. Installing local Python library. Put your dependencies into *DAG_FOLDER/libs* directory. Airflow automatically adds *DAG_FOLDER* and *PLUGINS_FOLDER* to the *PATH*. +### Azure +To deploy the Ingestion DAGs to airflow, follow below steps. +- Identify the file share to which DAGs need to be copied. [This](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/infra/templates/osdu-r3-mvp/service_resources/airflow.tf#L71) is the file share where DAGs for airflow reside. +- Copy contents of */src/dags* to *airflowdags/dags* folder +- Copy contents of */src/plugins/hooks* to *airflowdags/plugins/hooks* +- Copy contents of */src/plugins/operators* to *airflowdags/plugins/operators* + +#### Installing Python Dependencies +Python dependencies can be specified as extra pip packages in airflow deployment [here](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/airflow/helm-config.yaml#L211) + +#### Environment Variables & Airflow Variables +Add variables manually in the Airflow UI or through airflow helm charts. [List of the required variables](#required-variables). + +Adding variables to helm charts can be found [here](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/airflow/helm-config.yaml#L157) + +More details about airflow variables can be found [here](https://airflow.apache.org/docs/apache-airflow/1.10.12/concepts.html?highlight=airflow_var#variables) + ## DAG Implementation Details