Ingestion Workflow issueshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues2021-10-20T09:06:08Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/130Bump os-core-common.version to 0.12.0-rc32021-10-20T09:06:08ZMaksim MalkovBump os-core-common.version to 0.12.0-rc3Same as here
https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/181Same as here
https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/181M9 - Release 0.12Maksim MalkovMaksim Malkovhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/129Fixed get workflow status request by workflowName2021-10-11T09:44:52ZRiabokon Stanislav(EPAM)[GCP]Fixed get workflow status request by workflowNameGET "/{workflow_name}/workflowRun/{runId}" request used 'dagName' instead of 'workflowName' underhood during airflow call.
In case of different 'dagName' and 'workflowName' it leads error:
```
{
"code": 404,
"reason": "Failed to send...GET "/{workflow_name}/workflowRun/{runId}" request used 'dagName' instead of 'workflowName' underhood during airflow call.
In case of different 'dagName' and 'workflowName' it leads error:
```
{
"code": 404,
"reason": "Failed to send request.",
"message": "Unable to send request to Airflow. 404 NOT FOUND_{\"error\":\"Dag id workflow_name not found in DagModel\"}_"
}
```M9 - Release 0.12Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/118ADR : New API to handle System workflows2023-12-15T05:47:08Zpreeti singh[Microsoft]ADR : New API to handle System workflows**Context:**
===
System workflows are the workflows which are available to all the data partitions. Any System workflow can be triggered and retrieved by any of the tenants, but it can be Created, updated, deleted by only a user with spe...**Context:**
===
System workflows are the workflows which are available to all the data partitions. Any System workflow can be triggered and retrieved by any of the tenants, but it can be Created, updated, deleted by only a user with special privilege (Let’s say it has system role).
This is more with respect to the workflows or DAGs that OSDU provides with.
**How it's done today:**
===
- There is no concept of system workflows.
- The workflow metadata is stored in partition specific cosmos collection.
**Issue with current design:**
===
- The behavior of create api endpoint will change and can be confusing to users if we use same for system as well as private workflow. Users might end up unknowingly creating system workflow by passing data-partition-id of special partition.
- It is difficult for the updates to be managed for the changes from the OSDU community, if we try to copy or replicate the information across all the customer partitions.
**Proposal:**
===
There are two types of workflows in the system, System workflows and Private workflows. The proposal is to create a new API endpoint to register System workflows.
- The new API shall be termed as `workflow/system`
- To **create/update/delete** System workflows - `/workflow/system` endpoint shall be used
- To **Get/Trigger** System workflows, existing workflow service endpoint must be used.
- The authorization of new end point shall be different from existing groups. We'll use service principal based authorization.
- The new API shall not accept data-partition-id as a header. Service would be aware where the System workflows are located.
- This API should interact *only* with System workflows. It should not have access to other workflows.
**Sequence Diagram for createWorkflow**
![createWorkflow](/uploads/da01a45cf14062aad0a5cfc48bd51c3d/createWorkflow.png)
**Sequence Diagram for getallWorkflows**
![getallWorkflows](/uploads/f767cc09ea1b27baeb6137e6dbdd9959/getallWorkflows.png)
**Sequence Diagram for getWorkflow option 1** (this one got finalized)
![getWorkflowOption1](/uploads/55dcd0f4ece4c0df83f5bef057c1cfbb/getWorkflowOption1.png)
**Sequence Diagram for getWorkflow option 2**
![getWorkflowOption2](/uploads/5ee667cb7226a62a05a03a8effa27988/getWorkflowOption2.png)M9 - Release 0.12preeti singh[Microsoft]preeti singh[Microsoft]