Airflow 2.0 WIP
Tracking the task list for the Airflow 2.0 port described here.
Install Airflow 2.0 to all environments
Azure
-
Not Started -
In Progress -
Blocking -
Completed
AWS
-
Not Started -
In Progress -
Blocking -
Completed
GCP
-
Not Started -
In Progress -
Blocking -
Completed
IBM issue M9
-
Not Started -
In Progress -
Blocking -
Completed
Airflow 2.0 required DAG code changes (level of effort estimated at 5-days)
The strategy is for (EPAM) to deliver code changes for the manifest based ingestion DAGs issue and the WITSML parser issue which will demonstrate the required changes to the Python code to support both Airflow 2.+ and a deprecating strategy to continue support for Airflow 1.10.X.
2.1 Manifest-based ingestion (Python) [issue] @Kateryna_Kurach (#60 (closed))
-
Not Started -
In Progress - Planned delivery tagged for M7 -
Blocking -
Completed
2.2 WITSML parser (Python) issue @Kateryna_Kurach
-
Not Started -
In Progress - Planned delivery tagged for M7 -
Blocking -
Completed
2.3 SEGY -> OpenVDS - Planned delivery tagged for M8
-
Not Started -
In Progress -
Blocking -
Completed
-
Not Started -
In Progress -
Blocking -
Completed
CSV Parser issue @tdixon @sdubey7 @frubio (M8 by EPAM)
-
Not Started -
In Progress -
Blocking -
Completed
Workflow Services (level of effort estimated at 10-days)
Plans for updating the workflow services to be compatible with Airflow 2.0 is detailed here issue. (EPAM) has taken on this work.
Implement Aifrlow 2.0 stable API in workflow-core, make it optional.
-
Not Started - Planned delivery tagged for M8M9 -
In Progress -
Blocking -
Completed
Azure Migration @kibattul delivery M9
-
Not Started -
In Progress -
Blocking -
Completed
AWS Migration @Wibben
-
Not Started -
In Progress -
Blocking -
Completed
GCP Migration @Kateryna_Kurach M9
-
Not Started -
In Progress -
Blocking -
Completed
IBM Migration @shrikgar M8
-
Not Started -
In Progress -
Blocking -
Completed
4. Testing and pre-shipping
The dev team (??) will need to provide clear instructions for the Airflow upgrade which the CSP can follow in both QA and pre-ship environments so that it is prepped for testing by respective teams.
Create Airflow upgrade instructions.
-
Not Started -
In Progress -
Blocking -
Completed
QA - Airflow 2.+ readiness
-
Not Started -
In Progress -
Blocking -
Completed
Pre-shipping - Airflow 2.+ readiness
-
Not Started -
In Progress -
Blocking -
Completed