Ingestion Framework Guiding Principles

Ingestion Framework Guiding Principles

These guiding principles are shared by all the SLB authored user stories reflecting the capabilities of OpenDES

• Simple, Dedicated and Efficient APIs as ingestion entry points for each data format
• Store original high-fidelity data as is. Data in its original form must land first in the most appropriate store. Ex: - A DLIS file must land in the File DMS, A ZGY seismic survey must land in the Seismic DMS, etc. Once the data, in its original form, lands in the appropriate DMS, Parsers/Scanners/Enrichment processes can be applied to that data. The original data is stored as-is and parsers/scanner/clean-up processes will output derived entities.
• Framework should inherently enforce data lifecycle, Original ---> Well Known Structure ---> Well known Entity
• Extensibility of Parsers/Scanner/Enrichment processes through Configurations/Registrations
• Track flow and lifecycle of data in the data platform