ADR: Leverage File Service for Storage Operations
Introduction
Status
-
Initiated -
Proposed -
Under Review -
Approved -
Rejected
Decision
All DMS's will leverage the File Service as a layer between the DMS and storage. Additionally, the File Service will provide methods through the exposed service interface to move data on different storage tiers for all DMS's in the most cost-effective manner based on how it is being used. File Service will provide an abstraction over all storage actions, including calls to the partition service. Therefore, this will not need to be implemented in services that don't have that functionality yet. All DDMS's will use File Service for the storage of binary data, and other services will be able to leverage File Service as well.
This decision is a proposed solution to the rejection of this ADR
Add File Service Endpoints
Provide a new utility endpoint to retrieve the list of supported storage tiers. We will need to discuss how implement an endpoint that will return the list of supported storage tiers. Additionally, other functionality, such as finer-grained access which is required for SDMS storage procedures, will need to be included in File Service.
Refactor DMS Dataset Requests
Instead of directly loading in the Cosmos client library to each DMS, we will send the REST requests above to the file service to add datasets to the database.
Rationale
We want all capabilities regarding storage to be available for all DMS's with the smallest amount of variation possible. Additionally, by implementing these features once in one service (File Service), the community will save a lot of time because other services will not need to change when storage features change. The example we use above is storage tiers. Although this requires an initial investment in refactoring services to leverage File Service, we will ultimately be able to implement storage tiers for all DMS's without much change to the services themselves.
Consequences
We will need to:
- Add this additional functionality to the File Service
- "Onboard" services to using the File Service for all their storage actions
- Refactor all services to make REST requests to File Service as opposed to directly using the library
- We would need to enforce uniformity of requests given different services will be adding storage tier to their models
These tasks take a lot of time and effort as well as collaboration across many parties. We will need all CSV's and ISV's to support this motion and contribute to ensuring all services are compliant with this decision.