Modify delivery api to (optionally) support specialized cloud credentials
The delivery api supports a method "GetFileSignedUrl" which takes a list of srns and returns a list of signed urls. The behavior works in most cases but becomes a problem in two scenarios:
- Use of SDKs specific to a cloud providers where signed urls are not easily used
- Some cloud providers (AWS, IBM) do not support the ability to return signed urls for buckets or subpaths causing compatibility issues with clients that need this level access, like OpenVDS.
This request is to add support for optional data properties in the response of the "GetFileSignedUrl" api containing cloud specific security credentials to take advantage of alternative access methods to signed urls.
Related to R2 issue: https://gitlab.opengroup.org/osdu/r2-dev-issues/-/issues/11
There are two items below.
The first is an example of the response object for the delivery api in r1 which supported this approach. The second is a mocked response isolating the credentials into an extension area. These are included for discussion purposes
- Delivery api response from R1
{
"Result": [
{
"SRN": "srn:file/csv:1b8ca4b7dbb84b94ba41e51ff1e0eabd:1",
"Data": {
"GroupTypeProperties": {
"PreLoadFilePath": "s3://customer-1-demo-data-staging/data/provided/trajectory_csvs/1083.csv",
"TempWorkflowLocation": "s3://customer-1-demo-data-staging/data/provided/trajectory_csvs/1083.csv",
"FileSource": ""
},
"IndividualTypeProperties": {},
"ExtensionProperties": {}
},
"StorageLocation": {
"Bucket": "customer1-data-api-service-us-east-1-916427532483-data-res",
"Key": "files/srn_file!csv_1b8ca4b7dbb84b94ba41e51ff1e0eabd_1/file_object.csv"
},
"TemporaryCredentials": {
"SessionToken": "<session token>",
"AccessKeyId": "<access key>",
"Expiration": "2020-04-29T19:08:17+00:00"
}
}
],
"UnprocessedSRNs": []
}
- Example response object
{
"unprocessed": [
],
"processed": {
"srn:file/segy:3150654949254115366:": {
"signedUrl": "",
"unsignedUrl": "s3://osdu-seismic-test-data/volve/seismic/st0202/stacks/ST0202R08_PS_PSDM_RAW_PP_TIME.MIG_RAW.POST_STACK.3D.JS-017534.segy",
"kind": "opendes:osdu:file:0.2.0",
"PersistID": "B6AAB5922CD5257B"
},
"srn:file/segy:3150654949254115367:": {
"signedUrl": "",
"unsignedUrl": "s3://osdu-seismic-test-data/volve/seismic/st0202/stacks/ST0202R08_PS_PSDM_RAW_PP_TIME.MIG_RAW.POST_STACK.3D.JS-017534.segy",
"kind": "opendes:osdu:file:0.2.0",
"PersistID": "B6AAB5922CD5257C"
},
"srn:file/las2:30930288488995855085782:": {
"signedUrl": "https://minio-test-files.osdu-qa-a1c3eaf78a86806e299f5f3f207556f0-0000.us-south.containers.appdomain.cloud/osdu-seismic-test-data/r1/data/provided/well-logs/1226_dew05_1968_comp.las?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200526T204204Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=minio%2F20200526%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=8d413f4fe02b76d21cfc2a0c1a7835093651fc48f4b7c953570915cc724f4bb5",
"unsignedUrl": "s3://osdu-seismic-test-data/r1/data/provided/well-logs/1226_dew05_1968_comp.las",
"kind": "opendes:osdu:file:0.2.1"
}
},
"CredentialExtensions" : {
"Provider" : "IBM",
"IBMCredentials": {
"AccessKeyId": "9TD0A9ML4QFKECKA0EX3",
"SecretAccessKey": "FNh6fKb6tWjTR+vCFVdRvbWC9zdscLSh8B1G7fcq",
"SessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiI5VEQwQTlNTDRRRktFQ0tBMEVYMyIsImV4cCI6MzYwMDAwMDAwMDAwMCwicG9saWN5IjoiTmV3UG9saWN5In0.gO7pYBF_1vqhNYh6-PysaRPmcnDszZIjuWfXW5dbZauu8I-f8Ig2xhH4DyANIjtdtfdXqQ5NV2tAeJM-IXX2YQ",
"Expiration": "2020-05-22T21:48:53+00:00",
"Bucket":"osdu-seismic-test-data"
},
"AWSCrredentials" : {
},
"GCPCredentials" : {
},
"AzureCredentials" : {
}
}
}