Skip to content

Partition secret management should not be cloud vendor specific

Context & Scope

In current implementation of Reservoir DMS (M13 and M14), ReservoirDMS service needs information to connect to all PostgreSQL database (one database per data partition).

image

Edited after https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/raw/master/docs/images/architecture.png

In current Azure deployment of Reservoir DMS (since M13 and in M14), retrieving the connection string is not cloud-agnostic. On Azure, the connection string is stored in the Azure keyvault of the service resource group (with a data partition prefix).

  • Azure-specific code is generating an access token to this Azure keyvault
  • Azure-specific code uses this token to extract the reservoir connection secret (e.g. {DataPartitionID}-reservoir-conn) This is limiting deployment of Reservoir DMS on other cloud provider implementation, as Azure keyvault related code is not cloud agnostic. The final objective of this ADR is to get reservoir-connection secret for each data partition without relying on CSP-specific code. This ADR proposes a solution to remove Azure keyvault direct dependency to allow Reservoir DMS implementation by other CSPs.

Use Cases

  • I am a CSP devops. I need to deploy the Reservoir DMS service in any CSP implementation. The Reservoir DMS needs to connect to all my data partitions at start-up.

Requirements

  • Need to read a secret connection string to connect to PostgreSQL DB.
  • Secret connection string must be already written when deploying a PostgreSQL database per data partition.

Solution

In order to get the POSTRGRESQL connection string for specific partition, the server needs to access partition specific secrets. In the current Azure implementation, the access located in the common code, relies on Azure keyvault specific code. The server needs to replace this approach by either:

  • A common non-vendor-specific secret storage or API access
  • A mechanism where each cloud-vendor can implement its own partition secret management strategy

Solution 1: use existing OSDU Get Secret API Referring to this ADR: Secret Storage Service (osdu/platform/system/home#75 (closed)), we are in the configuration where secret information needs to be shared between services, and consequently we propose to use the associated Get Secret OSDU API (Retrieves the value of a secret from a specific repository by its key)

Solution 2: remove all C++ Azure code in folder Remove C++ code in the following folder https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/tree/main/src/lib/oes/providers/azure

Consequences

  • Azure deployment should be updated to account for Decision 1
  • Reservoir connection secret should be written to the OSDU Secret API
Edited by Chad Leong