README.md 5.21 KB
Newer Older
1
# Data Platform Core Services
2
3
4

## OSDU PMC Project Information

5
### Project Charter
6
To deliver and maintain the **core services** supporting the continous delivery of OSDU workflow and dataflow oriented services.
7

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
### Organization

Project Lead: @stephenwhitley 

**Voting Committers:**

The following are the voting committers for the Data Platform Core services project representing the resource commitment of Schlumberger and the Cloud Service Providers.
- @dkodeih, MSFT
- @fargyle, GCP 
- @babeal, AWS 
- @wladmirf. IBM 
- @ethiraj, SLB

**Maintainer Committers**

The Maintainer Committers are individuals within the Data Platform Core services project with the authority to approve Pull Requests and commit to Master.
- @ethiraj
- @babeal
- @sthende
- @alanbraz-ibm
- @danielscholl
- @renusood
30
- @mosiddi
31
- @Rudko
32
- @alrem
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

**Contributers**

As many as we can get, but primarily the developers that have been assigned from each of the cloud providers and Schlumberger.

* Business Analysts
  * @mcleminson - represents the product management of the contribution from Schlumberger
  * @gtmurphy - represents the business requirements of the contribution from Schlumberger
  * @ango2 - represents the business requirements of the contribution from Schlumberger
  * @doniger - represents the business requirements of the contribution from Shell
* Liaison with SLB Engineering Teams
  * @ChrisZhang - project manager for the same core services within Schlumberger
  * @hmarkovic - SLB Data Ecosystem lead architect


48
49
**Ways of Working**

50
51
For more detailed [Ways of Working](https://community.opengroup.org/osdu/platform/system/home/-/wikis/Governance), see the Wiki.

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

### Services

#### Services in the Systems Group

**[Storage Service](https://community.opengroup.org/osdu/platform/system/storage)** provides a set of APIs to manage the entire metadata life-cycle such as ingestion (persistence), modification, deletion, versioning and data schema. Storage Service is used to ingest metadata information generated by applications into the Data Ecosystem.

**[Indexer Service](https://community.opengroup.org/osdu/platform/system/indexer-service)** provides a mechanism for indexing documents that contain structured or unstructured data. Documents and indices are saved in a separate persistent store optimized for search operations. The indexer API can index any number of documents.

**[Indexer Queue Service](https://community.opengroup.org/osdu/platform/system/indexer-queue)** supports the above.

**[Search Service](https://community.opengroup.org/osdu/platform/system/search-service)** provides a model for indexing documents that contain structured data.

**[Schema Service](https://community.opengroup.org/osdu/platform/system/schema-service)** provides the ability to register new and validate against existing schemas to support new data types.

**[Delivery Service](https://community.opengroup.org/osdu/platform/system/delivery)** supports the return of Sign URL to cloud sources based on SRNs

**[Libraries](https://community.opengroup.org/osdu/platform/system/lib)** providing common functionality across all microservices

71
### Core Services that have been moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) Group
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

**[Legal Service](https://community.opengroup.org/osdu/platform/security-and-compliance/legal)** supports the creation and evaluation of legal (compliance) tags.  This project was moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) to capture new work in the area of dynamic policy evaluation.

**[Entitlements-ibm](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-ibm), [Entitlements-azure](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-azure), [Entitlements-aws](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-aws), and [Entitlements-gcp](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-gcp)** services are available for each cloud provide as there is little common code since this is primarily integration with Cloud specific identity services.

## Key Project Activities

**Note: R3 development workstreams for Core services**

1. [Port running services from R2 to GitLab](https://community.opengroup.org/osdu/documentation/-/wikis/OSDU-R2-code-freeze)
   *  Migrate from ADO the existing R2 services to Gitlab : `mostly complete`
   *  Get the R2 CI/CD pipeline working against Gitlab working with OG team : `mostly complete`
   *  Submit the delta changes (last 6 mo) from DES baseline into this baseline - for example search service fixes
1. Bring in the backlog of additional OpenDES contributions
   *  New services from DES baseline that were not in R2 - schema service, notification service, DDMS registry ...
   *  Define an SPI and port these to multiple cloud backends
   *  Adjust contributed test-cases to work against updated APIs for these services
   *  Add to CI/CD pipeline to make sure new services also build and can be deployed.
1. Enhanced core services with OSDU requirements
   * Community driven requirements such as policy-based entitlements etc.
   * Start off architects on elaboration on these tasks right away