Skip to content

M15 Feature-Batch records Fetching and added UT

Priyanka Bhongade requested to merge feature-dev-m15 into trusted-development

M15 Functionality

• Reference Data Mapping – Worked with the Data Definition team and created a new proposal and presented with weekly EDS call, Tested the solution with existing data • Let's assume we have 100 well at Data Provider ( AWS in this case ) whose created date is 12 Aug 22. We fetched 60 well into Data Operators (Azure in this case ). Now the LastSuccessfulRunDateUTC will update to 12 Aug 22, and after that, we run to fetch and ingest. Is that fetch the rest 40 wells? • Improve messaging, notification • Handling of failed attempts to ingest • Incorporate continuous feedback • EDS Pre ship test - fetch query does not respect wildcard syntax • Enable end user to configure schema authority variable at Data Operator side

M14 release branch from Community Master branch

Changes in import statement Modified code to access search, secret service and storage from Airflow URL instead of osdu_api.ini file which was deprecated Duplicated secret service in eds_scheduler as well because of the import issue Commented out unused secret service code Removed dependencies from osdu_api.ini on secret service requests

Merge request reports