... | ... | @@ -100,3 +100,55 @@ Other Workflow Engines \(e\.g\. Airflow\, Argo\, Azkaban\, etc\.\) |
|
|
|
|
|
![OSDUR3PlugnPlay](uploads/35c141e36fe8354de82e2ceac993b610/OSDUR3PlugnPlay.png)
|
|
|
|
|
|
# POC Findings
|
|
|
## Workflow Architecture Reviewed
|
|
|
|
|
|
### Side by Side Deployment
|
|
|
This picture shows the 3 implementations side-by-side and how each were architected and deployed on Azure:
|
|
|
|
|
|
![image](uploads/4d051924054cc889400112f48221c2ca/image.png)
|
|
|
|
|
|
### Workflow Steps
|
|
|
This picture show an example of the workflow steps executed and exercised in the POC:
|
|
|
![image](uploads/2e6fd30279d1e8dfca89602136ca06d2/image.png)
|
|
|
|
|
|
### Workflow Configuration
|
|
|
This picture shows how each workflow was configured in the 3 implementations:
|
|
|
![image](uploads/95c31106e22a14a184c8b6e4b2a2de10/image.png)
|
|
|
![image](uploads/e9af5c1cebe582b82ee876a2883dbb75/image.png)
|
|
|
![image](uploads/510847f6d0e04194e08dacd260b66917/image.png)
|
|
|
|
|
|
### Side-by-side NFRs and findings
|
|
|
|
|
|
![image](uploads/b4ffcd2c436dc2f8c543019dd41e8ece/image.png)
|
|
|
![image](uploads/9f238497df0eac01187f031fefa7a408/image.png)
|
|
|
|
|
|
# Performance Comparison - Breakdown
|
|
|
|
|
|
## Airflow Task Performance for Well Log Manifest Ingestion
|
|
|
|
|
|
- Total successful runs 9000+
|
|
|
- Overall Avg Duration: ~33 secs
|
|
|
- Airflow Avg Duration: ~32 secs
|
|
|
|
|
|
![image](uploads/59f1537e53f809bfb8f6607a859f79c7/image.png)
|
|
|
![image](uploads/fe21a617e17927bfa145e64d84d38003/image.png)
|
|
|
|
|
|
## ADFv2 Task Performance for Well Log Manifest Ingestion
|
|
|
|
|
|
- Total successful runs 9000+
|
|
|
- Overall Avg Duration: ~19 secs
|
|
|
- ADFv2 Avg Duration: ~16 secs
|
|
|
|
|
|
![image](uploads/4fa867de7301ce74d5fc71311794e825/image.png)
|
|
|
|
|
|
## Argo Task Performance for Well Log Manifest Ingestion
|
|
|
|
|
|
- Total successful runs 9000+
|
|
|
- Overall Avg Duration: ~22 secs
|
|
|
- Argo Avg Duration: ~21 secs
|
|
|
|
|
|
![image](uploads/ecbf0b9143d57ac6ebed4273c1675eab/image.png)
|
|
|
|
|
|
|
|
|
|