Explore projects

The Ingestion Workflow service provides a wrapper functionality around the Apache Airflow functions and is designed to carry out preliminary work with files before running the Airflow Directed Acyclic Graphs (DAGs) that will perform actual ingestion of OSDU data.

Docs: https://osdu.pages.opengroup.org/platform/data-flow/ingestion/ingestion-workflow

Repository for the Rock & Fluid Sample DDMS services for OSDU.

The Search Service supports full-text search on string fields, range queries on date, numeric or string fields, along with geo-spatial search.

Docs: https://osdu.pages.opengroup.org/platform/system/search-service

The Indexer Service indexes the metadata store to support the Search Service.

Docs: https://osdu.pages.opengroup.org/platform/system/indexer-service

The File Service provides internal and external APIs to request for file location data, such as a signed URL per file upload. Using the signed URL, OSDU users will be able to upload their files for ingestion into the system.

Docs: https://osdu.pages.opengroup.org/platform/system/file

The Notification service allows interested consumers to subscribe to data and metadata changes using a publish/subscriber pattern.

Docs: https://osdu.pages.opengroup.org/platform/system/notification

Secret Service facilitates the storage and retrieval of various types of secrets in a specified repository(ies) so that secrets can be secured, separated from the secrets in the infrastructure repository, and managed easily by interfacing applications.

Docs: https://osdu.pages.opengroup.org/platform/security-and-compliance/secret

The CRS Catalog Service enables end-users to make CRS selections, search for CRSs given a number of constraints, download of the entire catalog for local caching, and access to various sub-sets of the catalog.

The Storage Service handles metadata ingestion in OSDU.

Docs: https://osdu.pages.opengroup.org/platform/system/storage

This is the repository for automated QA tests done on the OSDU Data Platform for each milestone release with real-world datasets (TNO, Volve, etc.)

The Schema Service enables schema management in the OSDU and offers an implementation of the schema standard.

Docs: https://osdu.pages.opengroup.org/platform/system/schema-service

The Dataset Service provides internal and external APIs to allow fetching storage/retrieval instructions for various types of datasets.

Docs: https://osdu.pages.opengroup.org/platform/system/dataset

Legal service is to help ensure data compliance with legal data governance when dealing with data in the Data Ecosystem by managing legal tags

Docs: https://osdu.pages.opengroup.org/platform/security-and-compliance/legal

The Partition Service dynamically pulls the correct connection information at runtime to connect to the correct partition.

Docs: https://osdu.pages.opengroup.org/platform/system/partition

The Register Service manages subscriptions, action and DDMS registrations in the OSDU.

Docs: https://osdu.pages.opengroup.org/platform/system/register

Policy service is used for management and evaluation of dynamic policies in OSDU.

Docs: https://osdu.pages.opengroup.org/platform/security-and-compliance/policy

Airflow DAGs that consume Source Registry configuration to handle the fetch and ingest orchestration.

The project contains all Airflow dependencies need to OSDU.

Docs: https://osdu.pages.opengroup.org/platform/data-flow/ingestion/osdu-airflow-lib

The Unit Service provides dimension/measurement and unit definitions. Given two unit definitions, the service also offers conversion parameters in two different parameterizations.