[Delivery Service] add support for optional cloud credential connection string to support alternate access methods
Status
-
Proposed -
Trialing -
Under review -
Approved -
Retired
Context & Scope
The delivery api supports a method "GetFileSignedUrl" which takes a list of srns and returns a list of signed urls. The behavior works in most cases but becomes a problem in two scenarios:
- Use of SDKs specific to a cloud providers where signed urls are not easily used
- Some cloud providers (AWS, IBM) do not support the ability to return signed urls for buckets or subpaths causing compatibility issues with clients that need this level access, like OpenVDS.
This ADR is to add support for optional data properties in the response of the "GetFileSignedUrl" api containing cloud specific security credentials to take advantage of alternative access methods to signed urls.
Related to R2 issue: https://gitlab.opengroup.org/osdu/r2-dev-issues/-/issues/11
This issue is created to conform to the LADR template and to organize the discussion had here: https://community.opengroup.org/osdu/platform/system/delivery/-/issues/2
The proposed api is to add an optional connectionString
property to the api response that can be populated with credentials for alternate connection methods specific to the cloud provider. The exact content of this connection string is defined by each cloud provider.
An example of the proposed API response is shown below with examples responses for s3 and azure.
{
"unprocessed": [
],
"processed": {
"srn:vds:3150654949254115366:": {
"signedUrl": "",
"unsignedUrl": "s3://osdu-seismic-test-data/volve/seismic/st0202/stacks/vds/3150654949254115366",
"connectionString": "AccessKeyId=<access-key>;SecretAccessKey=<secret-key>;SessionToken=<session-token>;Expiration=2020-05-22T21:48:53+00:00;EndpointOverride=minio-test-files.osdu-qa-a1c3eaf78a86806e299f5f3f207556f0-0000.us-south.containers.appdomain.cloud",
"kind": "opendes:osdu:vds:0.2.0"
},
"srn:vds:3150654949254115366:": {
"signedUrl": "",
"unsignedUrl": "azure://osdu-seismic-test-data/volve/seismic/st0202/stacks/vds/3150654949254115366",
"connectionString": "DefaultEndpointsProtocol=https;AccountName=<account-name>;AccountKey=<account-key>;EndpointSuffix=core.windows.net",
"kind": "opendes:osdu:vds:0.2.0"
}
}
}
Other discussion points:
Today there are two different access models: single file and folder/bucket for OpenVDS repositories. The delivery service is responsible for determining which level of access to grant based on resolved srn "kind". In the future, if another "kind" needs a folder or bucket grant then the delivery service needs to be modified to add this kind to the list and redeployed. A non-embedded kind to access grant map api maybe needed in the future.
Decision
TBD
Rationale
Unblocks cloud providers that don't have the ability to support folder/bucket signed urls and doesn't break the current api.
Consequences
TBD
When to revisit
This decision precludes a broader discussion on the delivery api and signed urls that is planned to happen sometime in r3 and may need to be revisited then.
Tradeoff Analysis - Input to decision
Alternatives and implications
One alternative discussed was to expand the connection string properties and make them part of a credential object. There was some concern with this approach as it would lead to a dynamic api definition or a section of the api that wasn't defined in the schema. A single connection string property was agreed by all parties.
Decision criteria and tradeoffs
TBD
Decision timeline
Soon to unblock IBM and AWS from providing proper credentials for OpenVDS