Skip to content

Removal of Data Partition ID Dependency from Airflow Variables in EDS Scheduler DAG for Enhanced Multi-Partition Support

Shankar Pattanayak requested to merge eds-scheduler-data-partition-id-dependency-removal-form-airflow into master

Technical Debt User Story:

As an EDS team member, I need to remove the dependency on data partition IDs from Airflow variables. This will address technical debt by ensuring that eds_ingest and Naturalization DAGs can support multiple data partition IDs without disruption. In Milestone 24 (M24), we will focus on refactoring the eds_scheduler to eliminate this dependency and enhance multi-partition support.

Edited by Ashish Saxena

Merge request reports