Explore projects

The Time-series Storage Core (or internal) Service is a final step in time-series ingestion pipeline. Processes published ingestion messages and stores data to the Time-series Storage.

The Stream Storage Core (or internal) Service is a set of low level APIs for time-series data streams management.

Airflow DAGs that consume Source Registry configuration to handle the fetch and ingest orchestration.

The project contains all Airflow dependencies need to OSDU.

Docs: https://osdu.pages.opengroup.org/platform/data-flow/ingestion/osdu-airflow-lib

The Indexer Queue Service provides a set of APIs that help in forwarding the messages to and from the Storage Service and Indexer Service.

Docs: https://osdu.pages.opengroup.org/platform/system/indexer-queue

Airflow DAG for transformation from SEGY to OpenZGY

The Indexer Service indexes the metadata store to support the Search Service.

Docs: https://osdu.pages.opengroup.org/platform/system/indexer-service

This project provides Java client libraries for using OSDU core services.

The Ingestion Workflow service provides a wrapper functionality around the Apache Airflow functions and is designed to carry out preliminary work with files before running the Airflow Directed Acyclic Graphs (DAGs) that will perform actual ingestion of OSDU data.

Docs: https://osdu.pages.opengroup.org/platform/data-flow/ingestion/ingestion-workflow

Typescript client for open-etp-server, including REST API

This CSV Parser DAG supports the ingestion workflow for CSV type files

Library for integration with OSDU Core services

Library for shared components like httpclient, logging, units of measurement handling, etc.

Library for Messaging Queue SPI implementations

Library for Pekko http and streams common functionality

Library for common Redis functionality

Library for common security functionality