[M18] Authorization failure in EDS Ingest DAG

If authorization fails due to incorrect secret value, eds ingest fails by uncaught exception. Stack trace:

[2023-10-06, 13:00:27 UTC] {{src_dags_fetch_and_ingest.py:488}} ERROR - {'status': 'error', 'message': KeyError('value')}
Traceback (most recent call last):
  File "/usr/local/airflow/dags/eds_ingest/libs/src_dags_fetch_and_ingest.py", line 465, in _build_auth_code_request
    auth_code_authorization_code = self.generate_token.build_auth_code_authorization_code(self.auth)
  File "/usr/local/airflow/dags/eds_ingest/libs/generate_tokens.py", line 67, in build_auth_code_authorization_code
    value_data = self._get_secrets_values(secret_key)
  File "/usr/local/airflow/dags/eds_ingest/libs/generate_tokens.py", line 26, in _get_secrets_values
    secret_value = response_json['value']
KeyError: 'value'
Authorization can also cause the DAG to fail with other non-obvious errors:
  File "/usr/local/eds/airflow/dags/eds_ingest/libs/src_dags_fetch_and_ingest.py", line 88, in fetch_and_ingest
    connected_source_data_job = self.fetch_connected_source_registry_record.fetch_connected_source_data_job(
  File "/usr/local/eds/airflow/dags/eds_ingest/libs/fetch_registry_entry_record.py", line 71, in fetch_connected_source_data_job
    return search_response['results'][0]
KeyError: 'results'

Only after adding additional logging to the code the reason of the error can be found:

[2023-10-06, 16:58:00 UTC] {fetch_registry_entry_record.py:70} INFO - SERACH RESPONSE: {'reason': 'Access denied', 'status': 401, 'message': 'The user is not authorized to perform this action. Invalid token.'}
Edited by Dominik Kolasa