OSDU Ingestion
Review comments for R2 Orchestration and Ingestion Spike
- The workflow service is too broadly defined. Ex:- A LAS file has the following entities Wellbore, Log Sets, Log Headers, and Logs. Wellbore, Log Sets, and Log Headers are metadata information stored in Storage Service whereas Logs are bulk data stored in a Log Store. The LAS parsing workflow service has to deal with the above-defined entities. The ability to track the status of each of these tasks in the workflow and the ability to retry some of the failed tasks in the workflow is not clear in the design.
- Whenever each entity is stored in storage service, indexing service has to be explicitly called?