F1 (Spike) Story 18 Investigate core OSDU services that must be upgraded to support WIP resources in x-collaboration namespaces
Description
This is an umbrella spike investigation. We need to understand what core OSDU services (at least Storage, Indexer, Search, and Notification) must be upgraded to allow us to implement what is included in the P&WS MVP1 requirements for P&WS service.
As a short definition, what we require from those services is the so-named "x-collaboration header" (or in other words "custom namespaces") support.
It is needed, in particular, for the CPs' WIP resources inventory feature (see the Story #72 for "/projects/{id}/wip-resources API endpoint" implementation).
Here is what support is needed from each of the OSDU services:
-
Storage. It must support storing records in custom namespaces. We know that some part of the work is already done - the Storage service API (at least its Swagger) is upgraded for understanding the "x-collaboration" request header. However, we doubt if the header information is already respected by the CSPs' service/persistence layers. We need to test at least for AWS CSP and figure out the technical debt.
-
Indexer. When the Storage svc ingests a record it sends a "record_changed" event into the accordant OSDU's Pub/Sub topic. Then the Indexer picks up the message and performs indexing of the record into an accordant ES index. We need to check if the Indexer is already capable of distinguishing SOR and custom namespaces and indexing them into different indexes.
-
Search. When we send a query to the Search service we want it to distinguish requests for records in SOR or custom namespaces. For that, it must understand and respect "x-collaboration" headers.
-
Notification. This service subscribes to OSDU Pub/Sub topics and works as a proxy for outer services and applications that are its subscribers. We need to check if it already allows its subscribers to subscribe to only SOR or custom namespaces' records updates.
Acceptance criteria
For each point of the investigation, a separate ticket should be created and assigned to Dev team members. The current task should be closed only when:
- the results of all investigations are collected,
- the technical debt defined,
- and tickets for services' upgrades created.
Testing scenarios
Will be defined in the children tickets.
Technical notes
- for each investigation, clone repos and research the services documentation and source code: