Partition Service to Support Multiple Data Partitions

Overview

As part of the effort to enable multiple data partitions in OSDU, there is a need for a Partition Service that is responsible for creating and retrieving the partition specific properties on behalf of calling services. The service will encapsulate the data currently held in the secrets store and the "tenantinfo" datastore. Encapsulation in this case is referring to isolation via the service interface vs. any implication of where or how the secrets data, in particular, is physically stored.

The Partition Service will be the means of decoupling services from the logic of partition creation/info retrieval/deletion and will be fairly generic, i.e. essentially a means to store and retrieve key/value pair information relevant to data partitions. As a service rather than a client library, the Partition Service also provides a logical point to implement features related to performance and scalability. Additionally, the Partition Service will be language independent and available to all services without a separate implementation for each language family.

Details

For the Azure implementation of Partition Service, the following should hold:

• This will be a service that can only be accessed by any other services within the (multi-cluster) service mesh.
• It will have a 5-minute TTL on GET data partition info responses
• It will allow dirty reads if TTL has expired but can’t be updated by the client
• This should be implemented to the same standards as other OSDU services (technology stack, SPIs etc.) but with an Azure implementation first. This forces the interface of the API (partitionInfo) to be more generic.
• All data will be stored in Azure Key Vault whether this is secret or not. This means no partition data in CosmosDb.
• Service principal credentials are needed to access create and delete APIs. Get API should only be accessible within the cluster and has no public access outside the cluster.