Update OSDU Data Platform Data Loading Quick Start Guide authored by Débora Barretto's avatar Débora Barretto
...@@ -16,14 +16,6 @@ These are the important links/documents that, you should first try to read and b ...@@ -16,14 +16,6 @@ These are the important links/documents that, you should first try to read and b
2. [OSDU Data Definitions](https://community.opengroup.org/osdu/data/data-definitions) - This page contains the latest data definitions schema and reference values. 2. [OSDU Data Definitions](https://community.opengroup.org/osdu/data/data-definitions) - This page contains the latest data definitions schema and reference values.
3. [OSDU Core Services](https://community.opengroup.org/groups/osdu/platform/-/wikis/Core-Services-Overview) - This contains the latest core services API and documentation that the OSDU platform supports. 3. [OSDU Core Services](https://community.opengroup.org/groups/osdu/platform/-/wikis/Core-Services-Overview) - This contains the latest core services API and documentation that the OSDU platform supports.
## Overview
The OSDU Data Platform is versatile and designed to support multiple data loading use cases. The approaches recommended in this document are meant to offer a perspective for data ingestion. This document does not intend to prescribe the only path to data ingestion, and the approach provided is illustrative of some of the platform capabilities. We encourage you to engage with the OSDU member community with questions and feedback.
This data loading guide attempts to describe the latest practices for ingesting data into the OSDU Data Platform. The contents are intended to be fast-changing and evolving as the data loading capabilities of the platform are always updating. Once the workflows are matured, it will then be updated and reflected on the official documentation.
This document addresses end-to-end data loading from the perspective of the end-user, which in most cases is a member of the information management or data platform capabilities team. Hence, this guide assumes that this end-user has some basic technical knowledge regarding HTTP Web API, JSON data structure, and Python.
## Terms and Acronyms ## Terms and Acronyms
| Term | Description | | Term | Description |
|------|-------------| |------|-------------|
...@@ -32,6 +24,14 @@ This document addresses end-to-end data loading from the perspective of the end- ...@@ -32,6 +24,14 @@ This document addresses end-to-end data loading from the perspective of the end-
| Source Data | Source Data might be an Excel file, LAS/DLIS files, Seismic data, text files, data streams, databases, etc. One of the goals of OSDU is to support storing source data in its original format to preserve lineage. Metadata is created to allow this source data to remain in its original format yet remain searchable and discoverable within the platform. | | Source Data | Source Data might be an Excel file, LAS/DLIS files, Seismic data, text files, data streams, databases, etc. One of the goals of OSDU is to support storing source data in its original format to preserve lineage. Metadata is created to allow this source data to remain in its original format yet remain searchable and discoverable within the platform. |
| DDMS | Domain Data Management Services. This provides a single consistent set of APIs and methods to access the data objects regardless of the domain workflow. Here's a [list of the DDMS](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services) currently being developed. | | DDMS | Domain Data Management Services. This provides a single consistent set of APIs and methods to access the data objects regardless of the domain workflow. Here's a [list of the DDMS](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services) currently being developed. |
## Overview
The OSDU Data Platform is versatile and designed to support multiple data loading use cases. The approaches recommended in this document are meant to offer a perspective for data ingestion. This document does not intend to prescribe the only path to data ingestion, and the approach provided is illustrative of some of the platform capabilities. We encourage you to engage with the OSDU member community with questions and feedback.
This data loading guide attempts to describe the latest practices for ingesting data into the OSDU Data Platform. The contents are intended to be fast-changing and evolving as the data loading capabilities of the platform are always updating. Once the workflows are matured, it will then be updated and reflected on the official documentation.
This document addresses end-to-end data loading from the perspective of the end-user, which in most cases is a member of the information management or data platform capabilities team. Hence, this guide assumes that this end-user has some basic technical knowledge regarding HTTP Web API, JSON data structure, and Python.
## Introduction ## Introduction
The Data Flow workstream covers the full end-to-end process of facilitating data into the OSDU platform. The Data Ingestion workstream is part of the Data Loading workstream. The Data Flow workstream covers the full end-to-end process of facilitating data into the OSDU platform. The Data Ingestion workstream is part of the Data Loading workstream.
... ...
......