@@ -53,12 +55,14 @@ Environment dependencies might be installed by several ways:
...
@@ -53,12 +55,14 @@ Environment dependencies might be installed by several ways:
2. Setting up an environment into the Cloud Composer Console.
2. Setting up an environment into the Cloud Composer Console.
3. Installing local Python library. Put your dependencies into *DAG_FOLDER/libs* directory. Airflow automatically adds *DAG_FOLDER* and *PLUGINS_FOLDER* to the *PATH*.
3. Installing local Python library. Put your dependencies into *DAG_FOLDER/libs* directory. Airflow automatically adds *DAG_FOLDER* and *PLUGINS_FOLDER* to the *PATH*.
## DAG Implementation Details
## DAG Implementation Details
OSDU DAGs are cloud platform-agnostic by design. However, there are specific implementation requirements by cloud
OSDU DAGs are cloud platform-agnostic by design. However, there are specific implementation requirements by cloud
platforms, and the OSDU R2 Prototype provides a dedicated Python SDK to make sure that DAGs are independent from the
platforms, and the OSDU R2 Prototype provides a dedicated Python SDK to make sure that DAGs are independent from the
cloud platforms. This Python SDK is located in a separate [os-python-sdk] folder.
cloud platforms. This Python SDK is located in a separate [os-python-sdk] folder.
## Required Variables
## Required Variables
### Internal Services
### Internal Services
Some of the operators send requests to internal services. Hosts and endpoints are sepcified into Airflow Variables.
Some of the operators send requests to internal services. Hosts and endpoints are sepcified into Airflow Variables.
...
@@ -79,6 +83,7 @@ Some of the operators send requests to internal services. Hosts and endpoints ar
...
@@ -79,6 +83,7 @@ Some of the operators send requests to internal services. Hosts and endpoints ar
| provider | Need to properly initialize OSDU SDK |
| provider | Need to properly initialize OSDU SDK |
|entitlements_module_name | Need to properly initialize OSDU SDK |
|entitlements_module_name | Need to properly initialize OSDU SDK |