Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • H Home
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 15
    • Issues 15
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe SoftwareOpen Subsurface Data Universe Software
  • Platform
  • Data Flow
  • Data IngestionData Ingestion
  • Home
  • Issues
  • #18
Closed
Open
Issue created Jun 15, 2020 by Stephen Whitley (Invited Expert)@stephenwhitley

Ingestion Framework Guiding Principles

Ingestion Framework Guiding Principles

These guiding principles are shared by all the SLB authored user stories reflecting the capabilities of OpenDES

  • Simple, Dedicated and Efficient APIs as ingestion entry points for each data format
  • Store original high-fidelity data as is. Data in its original form must land first in the most appropriate store. Ex: - A DLIS file must land in the File DMS, A ZGY seismic survey must land in the Seismic DMS, etc. Once the data, in its original form, lands in the appropriate DMS, Parsers/Scanners/Enrichment processes can be applied to that data. The original data is stored as-is and parsers/scanner/clean-up processes will output derived entities.
  • Framework should inherently enforce data lifecycle, Original ---> Well Known Structure ---> Well known Entity
  • Extensibility of Parsers/Scanner/Enrichment processes through Configurations/Registrations
  • Track flow and lifecycle of data in the data platform
Edited Jun 18, 2020 by Shreyas Mehta
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking