EDS DAGs using variables from airflow runtime and also having duplicate variables.
We found 2 issues that can be addressed:
- CORE__CONFIG__SCHEMA_AUTHORITY: This variable seems DAG specific, but EDS DAGs need this environment variable in airflow runtimes. These should be defined within DAG.
- We need to Avoid duplicating variables in airflow environment, the dags or any client library should contain logic to create service urls given the dns host. CORE__SERVICE__WORKFLOW__URL should be replaced with existing CORE__SERVICE__WORKFLOW__HOST in dags code