Data Ingestion issueshttps://community.opengroup.org/groups/osdu/platform/data-flow/ingestion/-/issues2023-03-09T18:15:53Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/home/-/issues/18Ingestion Framework Guiding Principles2023-03-09T18:15:53ZStephen Whitley (Invited Expert)Ingestion Framework Guiding Principles
## Ingestion Framework Guiding Principles
These guiding principles are shared by all the SLB authored user stories reflecting the capabilities of OpenDES
* Simple, Dedicated and Efficient APIs as ingestion entry points for each data f...
## Ingestion Framework Guiding Principles
These guiding principles are shared by all the SLB authored user stories reflecting the capabilities of OpenDES
* Simple, Dedicated and Efficient APIs as ingestion entry points for each data format
* Store original high-fidelity data as is. Data in its original form must land first in the most appropriate store. Ex: - A DLIS file must land in the File DMS, A ZGY seismic survey must land in the Seismic DMS, etc. Once the data, in its original form, lands in the appropriate DMS, Parsers/Scanners/Enrichment processes can be applied to that data. The original data is stored as-is and parsers/scanner/clean-up processes will output derived entities.
* Framework should inherently enforce data lifecycle, Original ---> Well Known Structure ---> Well known Entity
* Extensibility of Parsers/Scanner/Enrichment processes through Configurations/Registrations
* Track flow and lifecycle of data in the data platformM1 - Release 0.1