Partition service
Status
- Proposed
- Trialing
- Under review
- Approved
- Retired
Context & Scope
The OSDU data platform supports the concept of data partitions and the availability of multiple partitions for a single deployment.
A data partition is the highest level of data isolation within an OSDU deployment and keeps data separated from other data partitions.
Today the specific properties that relate to a partition are largely captured through the shared ‘TenantInfo’ data object. Other properties specific to a partition e.g. secrets are often stored in different key vaults or databases and also need to be retrieved.
These are then high-frequency usage oints in the system and we want to rationalize these concerns in a central service to help maintain the integrity of the system.
Decision
For R3 we will encapsulate the storage and retrieval of partition specific properties within its own service.
Rationale
This will encapsulate the storage and retrieval of partition specific information. This will decouple implementations away from shared databases improving maintainability.
We can also encapsulate the performance, reliability, and scaling needs behind this service. REtrieving partition information is a high frequency, single point of failure in the system so having each service duplicate this increases the risk of service availability.
Consequences
Deployments will need to switch to store the partition specific properties via the service.
Services will need to be switched to retrieve partition specific properties via the service.
Trade-off Analysis - Input to decision
An alternative approach is the current Java shared code implementation. Here we see that a large amount of the partition specific properties are retrieved in a single POJO called ‘TenantInfo’. However other partition specific properties like Elastic Search credentials are retrieved from secret stores. There may be other variations on where this data may be retrieved from.
All of these are single points of failure in the system that are retrieved on every request. If you cannot get these properties, the request cannot be completed for any service that supports the data-partition-id header. Therefore, each service must implement the logic to retrieve this information or at least each language/framework in OSDU needs a shared component for the most common properties.
These properties are needed on every request and therefore having multiple implementations make having a performant, elasticity and reliable implementation both less likely and harder to achieve. Similarly coupling all services to specific databases in the system makes maintainability worse as there is tighter coupling of the system so any changes at the data layer end up requiring cascading changes in multiple services. However, these implementations do already exist and are working today. There is an effort involved in both creating this service and then migrating the other services to use this instead.
Decision criteria and trade-offs
- Elasticity
- Performance
- Reliability
- Maintainability
- Cost of change