Skip to content
GitLab
Closed
Issue created by Ben Lasscock@blasscoc💬Maintainer17 of 68 checklist items completed17/68 checklist items

Airflow 2.0 WIP

Tracking the task list for the Airflow 2.0 port described here.

Install Airflow 2.0 to all environments

Azure

  • Not Started
  • In Progress
  • Blocking
  • Completed

AWS

  • Not Started
  • In Progress
  • Blocking
  • Completed

GCP

  • Not Started
  • In Progress
  • Blocking
  • Completed

IBM issue M9

  • Not Started
  • In Progress
  • Blocking
  • Completed

Airflow 2.0 required DAG code changes (level of effort estimated at 5-days)

The strategy is for (EPAM) to deliver code changes for the manifest based ingestion DAGs issue and the WITSML parser issue which will demonstrate the required changes to the Python code to support both Airflow 2.+ and a deprecating strategy to continue support for Airflow 1.10.X.

2.1 Manifest-based ingestion (Python) [issue] @Kateryna_Kurach (#60 (closed))

  • Not Started
  • In Progress - Planned delivery tagged for M7
  • Blocking
  • Completed

2.2 WITSML parser (Python) issue @Kateryna_Kurach

  • Not Started
  • In Progress - Planned delivery tagged for M7
  • Blocking
  • Completed

2.3 SEGY -> OpenVDS - Planned delivery tagged for M8

  • Not Started
  • In Progress
  • Blocking
  • Completed

SEGY -> ZGY issue @sacha

  • Not Started
  • In Progress
  • Blocking
  • Completed

CSV Parser issue @tdixon @sdubey7 @frubio (M8 by EPAM)

  • Not Started
  • In Progress
  • Blocking
  • Completed

Workflow Services (level of effort estimated at 10-days)

Plans for updating the workflow services to be compatible with Airflow 2.0 is detailed here issue. (EPAM) has taken on this work.

Implement Aifrlow 2.0 stable API in workflow-core, make it optional.

  • Not Started - Planned delivery tagged for M8 M9
  • In Progress
  • Blocking
  • Completed

Azure Migration @kibattul delivery M9

  • Not Started
  • In Progress
  • Blocking
  • Completed

AWS Migration @Wibben

  • Not Started
  • In Progress
  • Blocking
  • Completed

GCP Migration @Kateryna_Kurach M9

  • Not Started
  • In Progress
  • Blocking
  • Completed

IBM Migration @shrikgar M8

  • Not Started
  • In Progress
  • Blocking
  • Completed

4. Testing and pre-shipping

The dev team (??) will need to provide clear instructions for the Airflow upgrade which the CSP can follow in both QA and pre-ship environments so that it is prepped for testing by respective teams.

Create Airflow upgrade instructions.

  • Not Started
  • In Progress
  • Blocking
  • Completed

QA - Airflow 2.+ readiness

  • Not Started
  • In Progress
  • Blocking
  • Completed

Pre-shipping - Airflow 2.+ readiness

  • Not Started
  • In Progress
  • Blocking
  • Completed
Edited by Chris Zhang