ADR: Ingestion Service - R3 Schema support
Context
- Data Loading requirements to enforce validation of users request before pass control to DAG logic.
- Currently all validation logic implemented in DAG itself.
- It was decided to do this work post-R3, but still will keep Ingestion Service in R3 architecture as an entry point for Ingestion.
- To work with R3, Ingestion Service endpoints should be updated to support R3 schemas
/submitWithManifest.
Scope
- Add a new endpoint
/api/v2/submit/manifest that will support R3 Manifest format.
- Update Ingestion Service logic to work with updated API of Workflow service (see ADR ???)
- See OpenAPI Spec for details
Decision
- Introduce a new endpoint
- Code change in Ingestion Service > Core to work with updated API of Workflow service
Rationale
- Required change for E2E R3 Ingestion
Consequences
- Work will be done by GCP
- Most of the changes will be done is Ingestion Service > Core
- No CSPs involvement required as there is not SPI layer in Ingestion service
When to revisit
