Add support for storing data shared across tasks in workflow
Service name: Ingestion workflow
Prerequisites:
[YES] the service is passing integration tests locally using code from master branch.
[YES] the service pipeline has been created and tested to see that it works and passes integration tests.
[YES/NO] the service helm chart has been created and tested to see that it works and that the deployed service will pass integration tests.
Steps:
Infrastructure Onboarding
-
Data needs to be shared across tasks in workflow. Currently there is no container that is available that stores such data. Hence we need another container ( workflow-tasks-sharing
) that will be used to share the data across the tasks in workflow.
Issue #34 (closed) is currently tracking this Work Item.
Additionally we would need to generate signed URLs to allow access to the data present in the workflow-tasks-sharing
storage container. For this, a cosmos collections is needed to store the metadata such as "workflowid", "runid", "state" etc. for the data present in the storage container (for which the SAS tokens will be generated)
Issue #32 (closed) is currently tracking this Work Item.
-
Obtain approval for any infrastructure requirements. -
Implement any required infrastructure changes. -
Obtain approval for merge request(s) containing infrastructure changes.
Chart Onboarding
Integration Test Onboarding
Manual Onboarding
Automation Onboarding