EDS - M17 feature and fixes Testing on Azure Preship with detailed observation and findings
-
eds_scheduler run details after changes in constant file and including airflow utility - eds_scheduler dag link : https://osdu-ship.msft-osdu-test.org/airflow2/graph?dag_id=eds_scheduler
-
Logger for eds_ingest - Display message for sample record - https://github.com/ExxonMobil/osdu-platform/issues/318 -
Logger for eds_ingest - Display Osdu_ingest run ID explicitly in logs - https://github.com/ExxonMobil/osdu-platform/issues/318 -
Password Credential OAuth flow type - EDS - Include Password Credentials OAuth Flow Type #303 (closed) - eds_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/xcom?dag_id=eds_ingest&task_id=fetch_client&execution_date=2023-05-02T11%3A45%3A47%2B00%3A00 CSRE : opendes:master-data--ConnectedSourceRegistryEntry:KatalystTestingM17
-
Validate the expiry of refresh token and generate a new refresh token and update secret vault - https://github.com/ExxonMobil/osdu-platform/issues/324 - eds_ingest dag link :
-
EDS - Adding Dynamic Schema Authority for Kind of CSRE ,CSDJ and ExternalReferenceValueMapping from Airflow Variable - https://github.com/ExxonMobil/osdu-platform/issues/360 -
EDS - Raise exception when Airflow Variable not found or None - https://github.com/ExxonMobil/osdu-platform/issues/359 - eds_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/log?dag_id=eds_ingest&task_id=fetch_client&execution_date=2023-05-02T08%3A10%3A55%2B00%3A00 - -
EDS: Conversion of the ConnectedSourceDataPartitionID to OnIngestionDataPartitionID for Array Datatype - https://github.com/ExxonMobil/osdu-platform/issues/322 - OSDU Ingest Link:
-
Test on Non-OSDU compliant Provider for master and WPC data -
Master Data -- SeismicAcquisitionSurvey
-
eds_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/log?dag_id=eds_ingest&task_id=fetch_client&execution_date=2023-05-02T11%3A54%3A02%2B00%3A00
-
Osdu_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/xcom?dag_id=Osdu_ingest&task_id=provide_manifest_integrity_task&execution_date=2023-05-02T11%3A54%3A15.263543%2B00%3A00
-
Work Product Component -- issue with Wrapper - unable to provide data as "()" in the query
-
eds_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/log?dag_id=eds_ingest&task_id=fetch_client&execution_date=2023-05-02T12%3A00%3A21%2B00%3A00
-
Osdu_ingest dag link :
-
-
Test on OSDU Compliant Provider for master and WPC data - AWS M16 -
Master Data -- Well
-
eds ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/xcom?dag_id=eds_ingest&task_id=fetch_client&execution_date=2023-05-02T07%3A59%3A56%2B00%3A00
-
Osdu_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/xcom?dag_id=Osdu_ingest&task_id=process_single_manifest_file_task&execution_date=2023-05-02T08%3A00%3A28.724429%2B00%3A00
-
Work Product Component Data -- WellLog
-
eds ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/xcom?dag_id=eds_ingest&task_id=fetch_client&execution_date=2023-05-02T08%3A06%3A35%2B00%3A00
-
Osdu_ingest dag link : https://osdu-ship.msft-osdu-test.org/airflow2/xcom?dag_id=Osdu_ingest&task_id=process_single_manifest_file_task&execution_date=2023-05-02T08%3A06%3A48.542915%2B00%3A00
-
-
Ingest any record and it will be added with "NameAliases" like below which has the source record id - https://github.com/ExxonMobil/osdu-platform/issues/289 "NameAliases": [ { "AliasName": "odesprod:master-data--Basin:Kam2_02", "AliasNameTypeID": ":reference-data--AliasNameType:EDSConnectedSourceIdentifier:", "DefinitionOrganisationID": null } ]
*Removed data-partition-id to avoid osdu_ingestion error
-
Create provider specific External Reference Value Mapping - https://github.com/ExxonMobil/osdu-platform/issues/376