# Data Platform Core Services

## OSDU PMC Project Information

### Project Charter
To deliver the data flow services of OSDU supporting the ingestion and enrichment of data.

### Organization

Project Lead: @stephenwhitley 

**Voting Committers:**

The following are the voting committers for the Data Platform Core services project representing the resource commitment of Schlumberger and the Cloud Service Providers.
- @dkodeih, MSFT
- @fargyle, GCP 
- @babeal, AWS 
- @wladmirf. IBM 
- @ethiraj, SLB

**Maintainer Committers**

The Maintainer Committers are individuals within the Data Platform Core services project with the authority to approve Pull Requests and commit to Master.
- @ethiraj
- @babeal
- @sthende
- @alanbraz-ibm
- @danielscholl
- @renusood

**Contributers**

As many as we can get, but primarily the developers that have been assigned from each of the cloud providers and Schlumberger.

* Business Analysts
  * @mcleminson - represents the product management of the contribution from Schlumberger
  * @gtmurphy - represents the business requirements of the contribution from Schlumberger
  * @ango2 - represents the business requirements of the contribution from Schlumberger
  * @doniger - represents the business requirements of the contribution from Shell
* Liaison with SLB Engineering Teams
  * @ChrisZhang - project manager for the same core services within Schlumberger
  * @hmarkovic - SLB Data Ecosystem lead architect



### Services

#### Services in the Systems Group

**[Storage Service](https://community.opengroup.org/osdu/platform/system/storage)** provides a set of APIs to manage the entire metadata life-cycle such as ingestion (persistence), modification, deletion, versioning and data schema. Storage Service is used to ingest metadata information generated by applications into the Data Ecosystem.

**[Indexer Service](https://community.opengroup.org/osdu/platform/system/indexer-service)** provides a mechanism for indexing documents that contain structured or unstructured data. Documents and indices are saved in a separate persistent store optimized for search operations. The indexer API can index any number of documents.

**[Indexer Queue Service](https://community.opengroup.org/osdu/platform/system/indexer-queue)** supports the above.

**[Search Service](https://community.opengroup.org/osdu/platform/system/search-service)** provides a model for indexing documents that contain structured data.

**[Schema Service](https://community.opengroup.org/osdu/platform/system/schema-service)** provides the ability to register new and validate against existing schemas to support new data types.

**[Delivery Service](https://community.opengroup.org/osdu/platform/system/delivery)** supports the return of Sign URL to cloud sources based on SRNs

**[Libraries](https://community.opengroup.org/osdu/platform/system/lib)** providing common functionality across all microservices

### Core Services that have been move to the Security and Compliance Group

**[Legal Service](https://community.opengroup.org/osdu/platform/security-and-compliance/legal)** supports the creation and evaluation of legal (compliance) tags.  This project was moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) to capture new work in the area of dynamic policy evaluation.

**[Entitlements-ibm](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-ibm), [Entitlements-azure](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-azure), [Entitlements-aws](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-aws), and [Entitlements-gcp](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-gcp)** services are available for each cloud provide as there is little common code since this is primarily integration with Cloud specific identity services.

## Key Project Activities

**Note: R3 development workstreams for Core services**

1. [Port running services from R2 to GitLab](https://community.opengroup.org/osdu/documentation/-/wikis/OSDU-R2-code-freeze)
   *  Migrate from ADO the existing R2 services to Gitlab : `mostly complete`
   *  Get the R2 CI/CD pipeline working against Gitlab working with OG team : `mostly complete`
   *  Submit the delta changes (last 6 mo) from DES baseline into this baseline - for example search service fixes
1. Bring in the backlog of additional OpenDES contributions
   *  New services from DES baseline that were not in R2 - schema service, notification service, DDMS registry ...
   *  Define an SPI and port these to multiple cloud backends
   *  Adjust contributed test-cases to work against updated APIs for these services
   *  Add to CI/CD pipeline to make sure new services also build and can be deployed.
1. Enhanced core services with OSDU requirements
   * Community driven requirements such as policy-based entitlements etc.
   * Start off architects on elaboration on these tasks right away