[EDS Scheduler] - Not creating new token token when current token expires
The EDS Scheduler DAG encounters a 401 Unauthorized (JWT Expired) error when attempting to call the Partitions service. The root cause appears to be that the EDS Scheduler does not generate a new token once the current JWT expires.
How to reproduce -
- Make sure no other DAG is running until current token expires.
- Monitor the EDS Scheduler logs. You will observe the following error - [2025-06-03, 15:20:05 IST] {fetch_data_partition.py:41} ERROR - Error decoding JSON response: Expecting value: line 1 column 1 (char 0)
Note - If any other DAG runs after the token expires, it will generate a new token. And once a new token is available, the EDS Scheduler resumes normal operation and the error no longer appears. This behavior has been observed in the M24 version of the DAG as well.
I added some interceptor code to print out additional details about API call EDS Scheduler is making. This confirmed that 401 response(JWT Expired) is being thrown.
cc: @ango2 @teoshengpu @shreyasinha @AshishSaxenaAccenture @adeole
Edited by Sudesh Tagadpallewar
