Explore projects

The Manifest Ingestion DAG includes a Workflow Engine, an implementation of Apache Airflow, to orchestrate manifest/metadata ingestion via Storage Service

The Notification service allows interested consumers to subscribe to data and metadata changes using a publish/subscriber pattern.

Docs: https://osdu.pages.opengroup.org/platform/system/notification

OBM (Object to Blob) Mapper

Typescript client for open-etp-server, including REST API

OQM (Object to Queue) Mapper

This project provides Java client libraries for using OSDU core services.

GitLab activity monitoring for OSDU projects. Built for human insights while providing AI consumable data.

The project contains all Airflow dependencies need to OSDU.

Docs: https://osdu.pages.opengroup.org/platform/data-flow/ingestion/osdu-airflow-lib

Manifest-based Ingestion

OSM (Object to KV-Store) Mapper

OBM (Object to Blob) Mapper

The Partition Service dynamically pulls the correct connection information at runtime to connect to the correct partition.

Docs: https://osdu.pages.opengroup.org/platform/system/partition

The CSV Parser is a key component of pddms-csv-parser time-series bulk data ingestion workflow and main processing unit of corresponding Apache Airflow DAG

[In development]

This is the repository for automated QA tests done on the OSDU Data Platform for each milestone release with real-world datasets (TNO, Volve, etc.)

Repository for the Rock & Fluid Sample DDMS services for OSDU.