Enhance Reference Data Mapping as per XOM suggestion[Techdebt]
To improve the performance of reference mapping and provide a more simplified ExternalReferenceValueMapping record
-
To change the logic to get reference data in demand rather than load all into memory initially -
Scope will hold the CSRE id -
SimpleMap.ReferenceValueID will hold the target reference mapping value -
HasGlobalScope will be default considered as false, which means the reference data mapping is specific to a particular data provider -
HasGlobalScope = true, which means the Scope will not be considered and it NOT specific to a particular data provider -
New Schema ExternalReferenceValueMappingSet for new nested structure[require feasibility study in M26] -
Deployment of new Schema -
Implementation based on the new schema and use old schema for backward compatibility
MR for HasGlobalScope= true: osdu/platform/data-flow/ingestion/osdu-airflow-lib!355 (merged)
Edited by Jeyakumar Devarajulu