Update Core Services Overview authored by Dania Kodeih (Microsoft)'s avatar Dania Kodeih (Microsoft)
......@@ -25,15 +25,6 @@ These are the data platform services for OSDU that form the core set of services
|File| [API](https://community.opengroup.org/osdu/platform/system/file/-/blob/master/docs/file-service_openapi.yaml) | [Notes](tbd) | [File Repo](https://community.opengroup.org/osdu/platform/system/file) | | [File Azure Repo](https://community.opengroup.org/osdu/platform/system/file/-/tree/master/provider/file-azure) | | |
## Data Flow Services
## DDMS
OpenVDS Libraries
=======================
This is the repository for OpenVDS:
- [OpenVDS](https://community.opengroup.org/osdu/open-vds)
OSDU Data Platform Libraries
=======================
These are the convenience libraries that the data platform uses across different services. There is a core set of libraries (common behavior) as well as cloud platform specific versions that facilitate integration in their environment:
......@@ -44,61 +35,4 @@ These are the convenience libraries that the data platform uses across different
- [GitLab GCP Lib](https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp)
- [GitLab IBM Lib ](https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm)
OSDU Ingestion Framework Prototype
==================================
- [OSDU R2 Protoype](https://github.com/google/framework-for-osdu/tree/master/osdu-r2)
- [OpenAPI Endpoint Specifications](https://amer-demo28-osdudev.apigee.io/apis)
- [Ingestion Design](https://community.opengroup.org/osdu/documentation/-/wikis/OSDU-(C)/Design-and-Implementation/Ingestion-and-Enrichment-Detail/R2-Ingestion-Workflow-Orchestration-non-Spike)
- [Implementation Design](https://github.com/google/framework-for-osdu/tree/master/osdu-r2)
Provider Specific Caveats in R2
================================
The Azure implementation of delivery follows a different implementation pattern as a result of provider dependency which will be addressed in R3. The Delivery function posts to a Delivery endpoint, but is hosted in the os-storage microservice.
The current API endpoint supported for delivery on Azure is as follows: https://\<main_service_URI\>/delivery/GetFileSignedURL
Agreed input/output and SPI
The main API call called for delivery is getFileSignedURLs(). Passing in the list of SRNs, it expects a list of file signed URLs in the JSON format below (output). We're generalizing using a String for the response (for now) to avoid pitfalls of strongly typing the response and reduce re-work when more functionality is added.
#### Delivery::GetFileSignedURL::Post - Input
```
{
srns: ["srn1", "srn2", "srn3"] < R2: required
}
```
#### Delivery::GetFileSignedURL::Post - Output
```
{
"unprocessed": ["srn1", "srn2"],
"processed": {
"srn3": {
"signedUrl": "https://...", < R2: required
"unsignedUrl": "...", < R2: Not required. Future extension point
"kind": "..." < R2: Not required. Future extension point
}
}
}
```
#### SPI
```
public interface ISignedURLService {
String getFileSignedURLS(List<String> srns); < R2: JSON representation of the API Output
}
```
We propose attaching the delivery API to the storage service as another endpoint.
Example query Implemented in this Story:
```
{
"SRNS": [ "srn:file/csv:a88797521dd111eab9ceff4274137c5c:1", … ]
}
```
Example query phase 2 (used for reference):
```
{
"SRNS": ["srn:master-data/Well:8438:",
"srn:work-product-component/WellboreMarker:8438_csv:",
"srn:file/csv:a88797521dd111eab9ceff4274137c5c:1"]
}
```