in GC R3M20 environment EDS_Schedule and EDS ingest not working
EDS_ingest while fetching the data seems to be running into problem:
AIRFLOW_CTX_DAG_RUN_ID=m19:master-data--ConnectedSourceDataJob:AWSPreshipKTSep252023__2023-09-28T14:05:00+00:00 [2023-09-28, 14:16:46 UTC] {src_dags_fetch_and_ingest.py:75} INFO - fetch_and_ingest---start [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:74} INFO - In Fetch ConnectedSourceDataJob --Start [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:84} INFO - In Fetch ConnectedSourceDataJob --End [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:42} INFO - In Fetch ConnectedSourceRegistry Entry --Start [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:52} INFO - In Fetch ConnectedSourceRegistry Entry --End [2023-09-28, 14:16:46 UTC] {src_dags_fetch_and_ingest.py:87} INFO - ConnectedSourceRegistryEntry ID: m19:master-data--ConnectedSourceRegistryEntry:AWSPreship-KTSep252023 [2023-09-28, 14:16:46 UTC] {src_dags_fetch_and_ingest.py:88} INFO - ConnectedSourceDataJob ID: m19:master-data--ConnectedSourceDataJob:AWSPreshipKTSep252023 [2023-09-28, 14:16:46 UTC] {src_dags_fetch_and_ingest.py:167} INFO - Source Data Job---start [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:107} INFO - In Fetch ExternalReferenceValueMapping --Start [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:110} INFO - Provider Id AWSPreship-KTSep252023 [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:113} INFO - namespace_id m19:reference-data--ExternalCatalogNamespace:EDS:AWSPreship-KTSep252023: [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:116} INFO - external_reference_kind osdu:wks:reference-data--ExternalReferenceValueMapping:1.0.0 [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:126} INFO - KIND=osdu:wks:reference-data--ExternalReferenceValueMapping:1.0.0 QUERY=data.NamespaceID: "\m19:reference-data--ExternalCatalogNamespace:EDS:AWSPreship-KTSep252023:\"" [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:127} INFO - Search Response {'results': [], 'aggregations': [], 'totalCount': 0} [2023-09-28, 14:16:46 UTC] {fetch_registry_entry_record.py:128} INFO - In Fetch ExternalReferenceValueMapping --End [2023-09-28, 14:16:46 UTC] {src_dags_fetch_and_ingest.py:270} INFO - _source_data_job_kind---end [2023-09-28, 14:16:46 UTC] {fetch_client.py:27} INFO - Fetch Client -- Start [2023-09-28, 14:16:47 UTC] {secret_service_client.py:34} INFO - In SecrectService --Start [2023-09-28, 14:16:47 UTC] {gc_credentials.py:81} INFO - Got SA_file. [2023-09-28, 14:16:47 UTC] {gc_credentials.py:146} INFO - Refresh token. [2023-09-28, 14:16:47 UTC] {gc_credentials.py:154} INFO - Token refreshed. [2023-09-28, 14:16:47 UTC] {secret_service_client.py:40} INFO - In SecrectService --End [2023-09-28, 14:16:47 UTC] {secret_service_client.py:34} INFO - In SecrectService --Start [2023-09-28, 14:16:47 UTC] {gc_credentials.py:81} INFO - Got SA_file. [2023-09-28, 14:16:47 UTC] {gc_credentials.py:146} INFO - Refresh token. [2023-09-28, 14:16:47 UTC] {gc_credentials.py:154} INFO - Token refreshed. [2023-09-28, 14:16:47 UTC] {secret_service_client.py:40} INFO - In SecrectService --End [2023-09-28, 14:16:47 UTC] {secret_service_client.py:34} INFO - In SecrectService --Start [2023-09-28, 14:16:47 UTC] {gc_credentials.py:81} INFO - Got SA_file. [2023-09-28, 14:16:47 UTC] {gc_credentials.py:146} INFO - Refresh token. [2023-09-28, 14:16:48 UTC] {gc_credentials.py:154} INFO - Token refreshed. [2023-09-28, 14:16:48 UTC] {secret_service_client.py:40} INFO - In SecrectService --End [2023-09-28, 14:16:48 UTC] {secret_service_client.py:34} INFO - In SecrectService --Start [2023-09-28, 14:16:48 UTC] {gc_credentials.py:81} INFO - Got SA_file. [2023-09-28, 14:16:48 UTC] {gc_credentials.py:146} INFO - Refresh token. [2023-09-28, 14:16:48 UTC] {gc_credentials.py:154} INFO - Token refreshed. [2023-09-28, 14:16:48 UTC] {secret_service_client.py:40} INFO - In SecrectService --End [2023-09-28, 14:16:48 UTC] {check_oauth_flow_type.py:28} INFO - Auth Code Generated for RefreshTokenKeyName [2023-09-28, 14:16:48 UTC] {check_oauth_flow_type.py:29} INFO - _build_auth_code_request---end [2023-09-28, 14:16:48 UTC] {fetch_client.py:61} INFO - QUERY BODY:{'kind': 'osdu:wks:master-data--Basin:1.0.0', 'query': '* AND ((createTime: [2023-02-01T00:00:00 TO 2023-09-28T14:16:46]) OR (modifyTime: [2023-02-01T00:00:00 TO 2023-09-28T14:16:46]))', 'sort': {'field': ['createTime'], 'order': ['ASC']}, 'limit': 5} [2023-09-28, 14:16:48 UTC] {src_dags_fetch_and_ingest.py:152} ERROR - {'status': 'error', 'message': JSONDecodeError('Expecting value: line 1 column 1 (char 0)')} Traceback (most recent call last): File "/opt/python3.8/lib/python3.8/site-packages/osdu_airflow/eds/eds_ingest/src_dags_fetch_and_ingest.py", line 92, in fetch_and_ingest records = self.fetch_client.fetch_details(criteria_dict,self.auth) File "/opt/python3.8/lib/python3.8/site-packages/osdu_airflow/eds/eds_ingest/implementation/fetch/fetch_client.py", line 66, in fetch_details search_body = json.loads(search_response.content) File "/opt/python3.8/lib/python3.8/json/__init__.py", line 357, in loads return _default_decoder.decode(s) File "/opt/python3.8/lib/python3.8/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/opt/python3.8/lib/python3.8/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)