[EDS Scheduler] - Not creating new token token when current token expires

The EDS Scheduler DAG encounters a 401 Unauthorized (JWT Expired) error when attempting to call the Partitions service. The root cause appears to be that the EDS Scheduler does not generate a new token once the current JWT expires.

How to reproduce -

  1. Make sure no other DAG is running until current token expires.
  2. Monitor the EDS Scheduler logs. You will observe the following error - [2025-06-03, 15:20:05 IST] {fetch_data_partition.py:41} ERROR - Error decoding JSON response: Expecting value: line 1 column 1 (char 0)

Note - If any other DAG runs after the token expires, it will generate a new token. And once a new token is available, the EDS Scheduler resumes normal operation and the error no longer appears. This behavior has been observed in the M24 version of the DAG as well.

m24

I added some interceptor code to print out additional details about API call EDS Scheduler is making. This confirmed that 401 response(JWT Expired) is being thrown.

image

cc: @ango2 @teoshengpu @shreyasinha @AshishSaxenaAccenture @adeole

Edited Jun 03, 2025 by Sudesh Tagadpallewar
Assignee Loading
Time tracking Loading