Explore projects

Legal service is to help ensure data compliance with legal data governance when dealing with data in the Data Ecosystem by managing legal tags

Docs: https://osdu.pages.opengroup.org/platform/security-and-compliance/legal

The CSV Parser is a key component of pddms-csv-parser time-series bulk data ingestion workflow and main processing unit of corresponding Apache Airflow DAG

Venus community implementation of OSDU Register: core and core-plus, no cloud-provider code. Generated from upstream while the prototype runs; upstream is the source of truth (see FORK-NOTICE.md).

Venus community implementation of OSDU Partition: core and core-plus, no cloud-provider code. Generated from upstream while the prototype runs; upstream is the source of truth (see FORK-NOTICE.md).

Venus community implementation of OSDU Notification: core and core-plus, no cloud-provider code. Generated from upstream while the prototype runs; upstream is the source of truth (see FORK-NOTICE.md).

This is the repository for automated QA tests done on the OSDU Data Platform for each milestone release with real-world datasets (TNO, Volve, etc.)

The External Data Sources Data Management Service (EDS DMS) integrates with the Dataset Service to accept and proxy requests for obtaining retrieval instructions for externally resing datasets and proxy. The service is implemented in Java and provides an open API.

A common place for shared CI/CD Pipeline files, for use with all OSDU Platform projects.

The Register Service manages subscriptions, action and DDMS registrations in the OSDU.

Docs: https://osdu.pages.opengroup.org/platform/system/register

Entitlements service is used to enable authorization in OSDU Data Ecosystem. The service allows for the creation and management of groups.

Docs: https://osdu.pages.opengroup.org/platform/security-and-compliance/entitlements

The Manifest Ingestion DAG includes a Workflow Engine, an implementation of Apache Airflow, to orchestrate manifest/metadata ingestion via Storage Service

Secret Service facilitates the storage and retrieval of various types of secrets in a specified repository(ies) so that secrets can be secured, separated from the secrets in the infrastructure repository, and managed easily by interfacing applications.

Docs: https://osdu.pages.opengroup.org/platform/security-and-compliance/secret

Airflow DAGs that consume Source Registry configuration to handle the fetch and ingest orchestration.

The Ingestion Workflow service provides a wrapper functionality around the Apache Airflow functions and is designed to carry out preliminary work with files before running the Airflow Directed Acyclic Graphs (DAGs) that will perform actual ingestion of OSDU data.

Docs: https://osdu.pages.opengroup.org/platform/data-flow/ingestion/ingestion-workflow

This CSV Parser DAG supports the ingestion workflow for CSV type files

The Notification service allows interested consumers to subscribe to data and metadata changes using a publish/subscriber pattern.

Docs: https://osdu.pages.opengroup.org/platform/system/notification

The Schema Service enables schema management in the OSDU and offers an implementation of the schema standard.

Docs: https://osdu.pages.opengroup.org/platform/system/schema-service