Skip to content

Updated EDS ingest code with logging and exception

Manish Kumar Tiwari requested to merge clean-up-dev-branch into master

Please find the description as below:

  1. Removed secretmanager_client (Cloud Specific Secret Manager) This will be replaced by Secret Service.
  2. Fetching Connected Source Data Job and Connected Source Registry Entry from eds_ingest for making criteria dict.
  3. Added new properties in criteria dict i.e lastSuccessfulRunDateUTC and dataProviderName.
  4. Implemented incremental fetch logic in _fetch_client.
  5. Added print statement for fetched records for clarity.
  6. Implemented AuthCode Generation for 2 flow types.
  7. Implemented Exceptional Handling to hand Runtime errors.
  8. Created a new Constant.py file to read Constants and Secrets(workaround).

Merge request reports