README.md 5.36 KB
Newer Older
1
# Data Platform Core Services
2
3
4

## OSDU PMC Project Information

5
### Project Charter
6
To deliver and maintain the **core services** supporting the continous delivery of OSDU workflow and dataflow oriented services.
7

8
9
### Organization

Chris Zhang's avatar
Chris Zhang committed
10
Project Lead: @ChrisZhang
11
12
13
14
15
16

**Voting Committers:**

The following are the voting committers for the Data Platform Core services project representing the resource commitment of Schlumberger and the Cloud Service Providers.
- @dkodeih, MSFT
- @fargyle, GCP 
Joe's avatar
Joe committed
17
18
- @deshruch, AWS 
- @wladmirf, IBM 
19
20
21
22
23
24
- @ethiraj, SLB

**Maintainer Committers**

The Maintainer Committers are individuals within the Data Platform Core services project with the authority to approve Pull Requests and commit to Master.
- @ethiraj
Joe's avatar
Joe committed
25
- @deshruch, AWS
26
27
28
29
- @sthende
- @alanbraz-ibm
- @danielscholl
- @renusood
30
- @mosiddi
Dmitriy Rudko's avatar
Dmitriy Rudko committed
31
32

GCP/EPAM:
Dmitriy Rudko's avatar
Dmitriy Rudko committed
33
- @Dmitriy_Rudko 
Dmitriy Rudko's avatar
Dmitriy Rudko committed
34
- @rostislav.dublin
Dmitriy Rudko's avatar
Dmitriy Rudko committed
35
- @Oleksandr_Kosse
Dmitriy Rudko's avatar
Dmitriy Rudko committed
36
- @Stanislav_Riabokon
37

Dmitriy Rudko's avatar
Dmitriy Rudko committed
38

39
40
41
42
43
44
45
46
**Subcommittee Contacts**

- @anup4461  
- @Jane
- @joe-equinor 
- @vishal


47
48
49
50
51
52
53
54
55
56
**Contributers**

As many as we can get, but primarily the developers that have been assigned from each of the cloud providers and Schlumberger.

* Business Analysts
  * @mcleminson - represents the product management of the contribution from Schlumberger
  * @gtmurphy - represents the business requirements of the contribution from Schlumberger
  * @ango2 - represents the business requirements of the contribution from Schlumberger
  * @doniger - represents the business requirements of the contribution from Shell
* Liaison with SLB Engineering Teams
Chris Zhang's avatar
Chris Zhang committed
57
  * @Fabio - project manager for the same core services within Schlumberger
58
59
60
  * @hmarkovic - SLB Data Ecosystem lead architect


61
62
**Ways of Working**

63
64
For more detailed [Ways of Working](https://community.opengroup.org/osdu/platform/system/home/-/wikis/Governance), see the Wiki.

65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

### Services

#### Services in the Systems Group

**[Storage Service](https://community.opengroup.org/osdu/platform/system/storage)** provides a set of APIs to manage the entire metadata life-cycle such as ingestion (persistence), modification, deletion, versioning and data schema. Storage Service is used to ingest metadata information generated by applications into the Data Ecosystem.

**[Indexer Service](https://community.opengroup.org/osdu/platform/system/indexer-service)** provides a mechanism for indexing documents that contain structured or unstructured data. Documents and indices are saved in a separate persistent store optimized for search operations. The indexer API can index any number of documents.

**[Indexer Queue Service](https://community.opengroup.org/osdu/platform/system/indexer-queue)** supports the above.

**[Search Service](https://community.opengroup.org/osdu/platform/system/search-service)** provides a model for indexing documents that contain structured data.

**[Schema Service](https://community.opengroup.org/osdu/platform/system/schema-service)** provides the ability to register new and validate against existing schemas to support new data types.

**[Delivery Service](https://community.opengroup.org/osdu/platform/system/delivery)** supports the return of Sign URL to cloud sources based on SRNs

**[Libraries](https://community.opengroup.org/osdu/platform/system/lib)** providing common functionality across all microservices

84
### Core Services that have been moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) Group
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105

**[Legal Service](https://community.opengroup.org/osdu/platform/security-and-compliance/legal)** supports the creation and evaluation of legal (compliance) tags.  This project was moved to the [Security and Compliance](https://community.opengroup.org/osdu/platform/security-and-compliance) to capture new work in the area of dynamic policy evaluation.

**[Entitlements-ibm](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-ibm), [Entitlements-azure](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-azure), [Entitlements-aws](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-aws), and [Entitlements-gcp](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements-gcp)** services are available for each cloud provide as there is little common code since this is primarily integration with Cloud specific identity services.

## Key Project Activities

**Note: R3 development workstreams for Core services**

1. [Port running services from R2 to GitLab](https://community.opengroup.org/osdu/documentation/-/wikis/OSDU-R2-code-freeze)
   *  Migrate from ADO the existing R2 services to Gitlab : `mostly complete`
   *  Get the R2 CI/CD pipeline working against Gitlab working with OG team : `mostly complete`
   *  Submit the delta changes (last 6 mo) from DES baseline into this baseline - for example search service fixes
1. Bring in the backlog of additional OpenDES contributions
   *  New services from DES baseline that were not in R2 - schema service, notification service, DDMS registry ...
   *  Define an SPI and port these to multiple cloud backends
   *  Adjust contributed test-cases to work against updated APIs for these services
   *  Add to CI/CD pipeline to make sure new services also build and can be deployed.
1. Enhanced core services with OSDU requirements
   * Community driven requirements such as policy-based entitlements etc.
   * Start off architects on elaboration on these tasks right away