README.md 5.05 KB
Newer Older
1
# Data Platform Core Services
2
3
4

## OSDU PMC Project Information

5
### Project Charter
6
To deliver and maintain the **core services** supporting the continous delivery of OSDU workflow and dataflow oriented services.
7

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
### Organization

Project Lead: @stephenwhitley 

**Voting Committers:**

The following are the voting committers for the Data Platform Core services project representing the resource commitment of Schlumberger and the Cloud Service Providers.
- @dkodeih, MSFT
- @fargyle, GCP 
- @babeal, AWS 
- @wladmirf. IBM 
- @ethiraj, SLB

**Maintainer Committers**

The Maintainer Committers are individuals within the Data Platform Core services project with the authority to approve Pull Requests and commit to Master.
- @ethiraj
- @babeal
- @sthende
- @alanbraz-ibm
- @danielscholl
- @renusood
30
- @mosiddi
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

**Contributers**

As many as we can get, but primarily the developers that have been assigned from each of the cloud providers and Schlumberger.

* Business Analysts
  * @mcleminson - represents the product management of the contribution from Schlumberger
  * @gtmurphy - represents the business requirements of the contribution from Schlumberger
  * @ango2 - represents the business requirements of the contribution from Schlumberger
  * @doniger - represents the business requirements of the contribution from Shell
* Liaison with SLB Engineering Teams
  * @ChrisZhang - project manager for the same core services within Schlumberger
  * @hmarkovic - SLB Data Ecosystem lead architect



### Services

#### Services in the Systems Group

**[Storage Service](https://community.opengroup.org/osdu/platform/system/storage)** provides a set of APIs to manage the entire metadata life-cycle such as ingestion (persistence), modification, deletion, versioning and data schema. Storage Service is used to ingest metadata information generated by applications into the Data Ecosystem.

**[Indexer Service](https://community.opengroup.org/osdu/platform/system/indexer-service)** provides a mechanism for indexing documents that contain structured or unstructured data. Documents and indices are saved in a separate persistent store optimized for search operations. The indexer API can index any number of documents.

**[Indexer Queue Service](https://community.opengroup.org/osdu/platform/system/indexer-queue)** supports the above.

**[Search Service](https://community.opengroup.org/osdu/platform/system/search-service)** provides a model for indexing documents that contain structured data.

**[Schema Service](https://community.opengroup.org/osdu/platform/system/schema-service)** provides the ability to register new and validate against existing schemas to support new data types.

**[Delivery Service](https://community.opengroup.org/osdu/platform/system/delivery)** supports the return of Sign URL to cloud sources based on SRNs

**[Libraries](https://community.opengroup.org/osdu/platform/system/lib)** providing common functionality across all microservices

65
### Core Services that have been moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) Group
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

**[Legal Service](https://community.opengroup.org/osdu/platform/security-and-compliance/legal)** supports the creation and evaluation of legal (compliance) tags.  This project was moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) to capture new work in the area of dynamic policy evaluation.

**[Entitlements-ibm](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-ibm), [Entitlements-azure](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-azure), [Entitlements-aws](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-aws), and [Entitlements-gcp](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-gcp)** services are available for each cloud provide as there is little common code since this is primarily integration with Cloud specific identity services.

## Key Project Activities

**Note: R3 development workstreams for Core services**

1. [Port running services from R2 to GitLab](https://community.opengroup.org/osdu/documentation/-/wikis/OSDU-R2-code-freeze)
   *  Migrate from ADO the existing R2 services to Gitlab : `mostly complete`
   *  Get the R2 CI/CD pipeline working against Gitlab working with OG team : `mostly complete`
   *  Submit the delta changes (last 6 mo) from DES baseline into this baseline - for example search service fixes
1. Bring in the backlog of additional OpenDES contributions
   *  New services from DES baseline that were not in R2 - schema service, notification service, DDMS registry ...
   *  Define an SPI and port these to multiple cloud backends
   *  Adjust contributed test-cases to work against updated APIs for these services
   *  Add to CI/CD pipeline to make sure new services also build and can be deployed.
1. Enhanced core services with OSDU requirements
   * Community driven requirements such as policy-based entitlements etc.
   * Start off architects on elaboration on these tasks right away