... | ... | @@ -41,4 +41,7 @@ There are also recommended version standards for development. See this [ADR](htt |
|
|
|
|
|
* Astronomer guide for Airflow DAG authoring - [here](https://www.astronomer.io/guides/dag-best-practices#:~:text=%20DAG%20Writing%20Best%20Practices%20in%20Apache%20Airflow,its%20own%20container%20with%20limited%20memory...%20More%20)
|
|
|
* Apache best practices guide - not much, but a few pointers [here](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html)
|
|
|
* Jinja macros, testing tips - automation tests [here](https://towardsdatascience.com/best-practices-for-airflow-developers-990c8a04f7c6) |
|
|
\ No newline at end of file |
|
|
* Jinja macros, testing tips - automation tests [here](https://towardsdatascience.com/best-practices-for-airflow-developers-990c8a04f7c6)
|
|
|
|
|
|
** Cloud and Airflow Agnostic **
|
|
|
DAG operators must be cloud-agnostic. Leverage the OSDU Python SDK as a Service Provider Interface (SPI) to ensure the DAG operators don't have cloud-specific implementations. Additionally, each DAG operator must be Airflow agnostic. In other words, external code should be able to invoke the logic within the DAG operator without running inside the Airflow runtime or by including Airflow libraries. |
|
|
\ No newline at end of file |