EDS: Partition URL/HOST inconsistencies in different CSP

Current Scenario: Partition service is used to get the partition details upon execution of Schedular DAG run & Email Notification. The core_service_partition_host is defined in the airflow variables, and the complete URL is formed in EDS on run rime.

image

image

EDS is hardcoded to use partition service v1.

Issue/Refactoring: While this is working with CSP like GC where the version is not handled in the Airflow:

GC:

image

it errors in CI, as the partition_host url has the version defined, causing the scheduler to fail.

CI:

image

GLAB:

image.png

image.png

Impact: EDS Scheduler & Email Notification

Proposed fix:

Change EDS to use "core_service_partition_url", Remove the api version from endpoint in the EDS code as version should be configurable, and as seen in the past different CSPs can run on different versions of the same service. Additionally, adding better exception handling to improve the issue identification.

Airflow variables for partition service should be defined like this:

  core_service_partition_host: https://{{hostname}}
  core_service_partition_url: https://{{hostname}}/api/partition/v1
Edited by Shreya Sinha