ADR: Workflow Service Environment Standardization
Context
Providing consistent workflow runtime environments enables DAGs (Directed Acyclic Graphs) to be written once and run across any standardized workflow service environment. There are some differences in the Workflow Service environments built for R3, so we must agree on the versions of the major components of the Workflow Service to achieve standardization.
Scope
- All Workflow Service implementations should operate with the same
major.minorversion of Airflow. - All Workflow Service implementations should operate with the same
major.minorPython version within Airflow. - All Workflow Service DAG Operators should be authored to run with the same
major.minorPython version within Airflow.
Decision
Standardize on the following Workflow Service component versions
| Component | Version |
|---|---|
| Airflow | 1.10.x |
| Airflow Python Runtime | 3.6.x |
| DAG Operator Python Development Version | 3.6.x |
Rationale
- Workflows (DAGs) written against the standard will be portable to all standardized Workflow Service runtime environments.
Consequences
- Workflow Service implementers may have to change Airflow and Python versions and re-test developed workflows (DAGs)