GONRG-2913: Added support for whitelist reference patterns to exclude from... (!64) · Merge requests · OSDU Software / OSDU Data Platform / Data Flow / Data Ingestion / Manifest Ingestion DAG · GitLab

Aleksandr Spivakov (EPAM) requested to merge GONRG-2913_whitelist_reference_patterns_feature into master Aug 09, 2021

Type of change

Bug Fix
Feature

Does this introduce a change in the core logic?

[Yes]

Does this introduce a change in the cloud provider implementation, if so which cloud?

AWS
Azure
GCP
IBM

Updates description?

Introduces availability to whitelist references using custom regexp patterns to exclude those references from referential integrity validation.

New Airflow variable example:

See related MR for more details: osdu/platform/system/sdks/common-python-sdk!22 (merged)

closes https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/180

Edited Aug 09, 2021 by Siarhei Khaletski (EPAM)