Manifest Ingestion DAG merge requestshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests2023-08-18T11:13:27Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/105add all files for task airflow scheduler2023-08-18T11:13:27ZAliaksei Kruk2add all files for task airflow schedulerM12 - Release 0.15Oleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/34Added BaseTokenRefresher2023-08-18T11:14:37ZSiarhei Khaletski (EPAM)Added BaseTokenRefresher## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
- Added BaseTokenRefresherM5 - Release 0.8Siarhei Khaletski (EPAM)Siarhei Khaletski (EPAM)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/29Added readme for azure2023-08-18T11:14:42ZKishore BattulaAdded readme for azure## Type of change
- [] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [No]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] I...## Type of change
- [] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [No]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Updated documentation on how to deploy manifest ingestion DAGs into airflowM5 - Release 0.8https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/96Adding extra index urls to enable finding these packages2022-02-09T17:16:12ZDavid Diederichd.diederich@opengroup.orgAdding extra index urls to enable finding these packages## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- No
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IB...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- No
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Adding in the --extra-index-url options to the `requirements.txt` file so this can build out of the boxM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/162Add workflow rule for CI2023-09-25T08:25:53Zsaketh somarajuAdd workflow rule for CI## Type of change
- [ ] Bug Fix
- [ ] Feature
- [*] CI change
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [*] Azure
...## Type of change
- [ ] Bug Fix
- [ ] Feature
- [*] CI change
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [*] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
- Added workflow rules to configure release and tag pipelines to run on dev secondary environment.
- Related to https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/merge_requests/995M21 - Release 0.24saketh somarajusaketh somarajuhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/107Anthos bootstrap skip2022-04-21T12:36:12ZYan Sushchynski (EPAM)Anthos bootstrap skip## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [No]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] ...## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [No]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IBM
## Updates description?
Add skipp of Anthos bootstrap step and publish DAGs.M12 - Release 0.15https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/157apply changes from Azure/M16 to Azure/m182023-08-15T19:17:00ZNathan Strelserapply changes from Azure/M16 to Azure/m18## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] ...## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Merged changes in azure m16 branch to the Azure m18 branchNathan StrelserNathan Strelserhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/40Aws impl, id version bug fix2023-08-18T11:14:32ZSpencer Suttonsuttonsp@amazon.comAws impl, id version bug fix## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Aws implementation code.
Also bug fix that addresses this issue: https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/55M5 - Release 0.8Spencer Suttonsuttonsp@amazon.comSpencer Suttonsuttonsp@amazon.comhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/68Azure - Adding E2E Test Stage2023-08-18T11:14:01Zharshit aggarwalAzure - Adding E2E Test Stage## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ]...## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M8 - Release 0.11harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/63Azure - Adding Gitlab pipelines2023-08-18T11:14:04Zharshit aggarwalAzure - Adding Gitlab pipelines## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] ...## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Adding Azure pipeline, end_to_end_test_dag stage will be implemented in a subsequent MRM8 - Release 0.11harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/69Azure - Add Test Stage ADO pipeline2023-08-18T11:13:59Zharshit aggarwalAzure - Add Test Stage ADO pipelineM8 - Release 0.11harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/95Azure - Add zip file2023-08-18T11:13:35Zharshit aggarwalAzure - Add zip fileM10 - Release 0.13harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/22Azure credentials ut2023-08-18T11:14:51ZKishore BattulaAzure credentials ut## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Added Unit tests for azure_credentials.pyM4 - Release 0.7https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/77Azure - pipeline fix2023-08-18T11:13:51Zharshit aggarwalAzure - pipeline fix## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M9 - Release 0.12harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/71Azure - pipeline script fix2023-08-18T11:13:55Zharshit aggarwalAzure - pipeline script fix## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M8 - Release 0.11harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/80Azure Script Fix2023-08-18T11:13:49Zharshit aggarwalAzure Script Fix## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M9 - Release 0.12harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/74Azure - Update ADO pipeline2023-08-18T11:13:52Zharshit aggarwalAzure - Update ADO pipeline## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M9 - Release 0.12harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/73Azure - Update Test Dag stage2023-08-18T11:13:54Zharshit aggarwalAzure - Update Test Dag stage## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M9 - Release 0.12harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/65Azure - Updating ADO pipeline2023-08-18T11:14:02Zharshit aggarwalAzure - Updating ADO pipelineM8 - Release 0.11harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/98Cherry-pick 'Adding extra index urls to enable finding these packages' into '...2022-02-09T18:02:47ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Adding extra index urls to enable finding these packages' into 'release/0.13'Original MR: !96Original MR: !96M10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.org