Ingestion Workflow merge requestshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests2024-03-20T09:04:02Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/531java17migration2024-03-20T09:04:02ZVikas Hoode [BP]vikas.hoode@bp.comjava17migrationJdk 17 migrationJdk 17 migrationM23 - Release 0.26Vikas Hoode [BP]vikas.hoode@bp.comVikas Hoode [BP]vikas.hoode@bp.comhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/530Trusted java17migration2024-03-18T12:23:16ZVikas Hoode [BP]vikas.hoode@bp.comTrusted java17migrationjava 17 migrationjava 17 migrationM23 - Release 0.26Vikas Hoode [BP]vikas.hoode@bp.comVikas Hoode [BP]vikas.hoode@bp.comhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/520Update lib2024-02-12T21:04:55ZRucha DeshpandeUpdate libhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/519Draft: Updeate os-core-lib-aws version to remediate vulnerability2024-02-12T17:32:14ZSolomon AyalewDraft: Updeate os-core-lib-aws version to remediate vulnerabilityhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/513Merge branch 'api-extend-run-details' into 'master'2024-01-05T09:32:59ZRiabokon Stanislav(EPAM)[GCP]Merge branch 'api-extend-run-details' into 'master'Implement Airflow facade endpoint
See merge request osdu/platform/data-flow/ingestion/ingestion-workflow!505
(cherry picked from commit 2464a39b22f5dc9c984db45fe09b361409cefd93)
a26233b7 added api endpoint to fetch task details with x...Implement Airflow facade endpoint
See merge request osdu/platform/data-flow/ingestion/ingestion-workflow!505
(cherry picked from commit 2464a39b22f5dc9c984db45fe09b361409cefd93)
a26233b7 added api endpoint to fetch task details with xcom values
69dc8125 undo Stas test changes
ba6417ef move tests
61cb1746 Merge remote-tracking branch 'origin/master' into api-extend-run-details
fa5847d1 Merge remote-tracking branch 'origin/master' into api-extend-run-details
502f760e Merge remote-tracking branch 'origin/master' into api-extend-run-details
2f44c1c8 Added condition to /{workflow_name}/workflowRun/{runId}/latestInfoM22 - Release 0.25David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/509Draft: fix spring-security vulnerabilities2023-12-19T13:13:26ZVidyaDharani LokamDraft: fix spring-security vulnerabilitiesVidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/495Implement Airflow facade endpoint2023-12-13T15:21:59ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comImplement Airflow facade endpointhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/159
# How to test:
Postman and integration tests.
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainabilit...https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/159
# How to test:
Postman and integration tests.
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [x] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GC
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [x] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.M23 - Release 0.26Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/439Renamed domain env variable (GONRG-6912)2023-12-08T15:15:57ZYurii Ruban [EPAM / GCP]Renamed domain env variable (GONRG-6912)# Description:
Renamed the domain environment variable to group id for a better understanding of the origin of the variables
# How to test:
Check unit and int tests.
# Changes include:
- [ ] Refactor (a non-breaking change that impro...# Description:
Renamed the domain environment variable to group id for a better understanding of the origin of the variables
# How to test:
Check unit and int tests.
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
# Changes in:
- [ ] GCPRiabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/432Draft: Az/fix code coverage2023-12-08T15:15:55Zsaketh somarajuDraft: Az/fix code coveragehttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/481Pass workflow user ID to the Airflow as part of payload (GONRG-7795)2023-11-08T10:37:23ZRiabokon Stanislav(EPAM)[GCP]Pass workflow user ID to the Airflow as part of payload (GONRG-7795)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/157
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/158
Removed a custom header 'x-user-id' fro...https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/157
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/158
Removed a custom header 'x-user-id' from a core part of service.M22 - Release 0.25Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/484Draft: delete dag file2023-10-16T04:26:33Zsaketh somarajuDraft: delete dag filesaketh somarajusaketh somarajuhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/479Draft: Az/sa add sleep2023-10-12T15:29:45Zsaketh somarajuDraft: Az/sa add sleephttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/417File Service Resiliency Enhacements2023-10-11T16:15:16ZChahat KumarFile Service Resiliency Enhacementshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/358whitesource updates2023-08-25T22:21:36ZGauri Chitalewhitesource updateshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/377Squashed commit of the following2023-08-25T22:21:35ZManish JangidSquashed commit of the followingcommit 16a40124
Author: fhoueto.amz <fhoueto@amazon.com>
Date: Thu Jan 19 2023 13:31:04 GMT-0600 (Central Standard Time)
add roles to ingestion workflow open api speccommit 16a40124
Author: fhoueto.amz <fhoueto@amazon.com>
Date: Thu Jan 19 2023 13:31:04 GMT-0600 (Central Standard Time)
add roles to ingestion workflow open api spechttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/435Draft: change SA name2023-08-25T22:21:30ZOrest Khud [EPAMĀ / GCP]Draft: change SA nameM20 - Release 0.23https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/470aws-sync-dev-to-master2023-08-19T02:35:11ZLong Chengaws-sync-dev-to-masterM20 - Release 0.23Long ChengLong Chenghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/434Draft: upgrading corelibazure version2023-08-02T04:33:44ZJayesh BagulDraft: upgrading corelibazure versionhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/459Upgrade First Party Library Dependencies for Release 0.222023-07-18T07:07:35ZChad LeongUpgrade First Party Library Dependencies for Release 0.22This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 3b6262fcd211d4d32b407cd34b0f559693e4e759
Maven: 0.23.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | --------------------- |
| core-lib-azure | 0.22.0-rc2 | 0.0.28 |
| core-lib-gc | 0.21.0-rc5 | |
| os-core-lib-aws | 0.21.0-rc5 | 0.21.0-rc5 |
| obm | 0.21.0-rc2 | |
| oqm | 0.21.0-rc4 | |
| os-core-common | 0.19.0-rc8, 0.21.0-rc5 | 0.13.0, 0.20.1 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.8.0 |
| osm | 0.21.0-rc3 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.5, 2.13.2.2, 2.13.4.2 | 2.13.2.2, 2.10.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.3.28, 5.3.24, 5.3.22 | 5.2.2.RELEASE, 5.3.12 |
| (3rd Party) org.yaml.snakeyaml | 2.0 | 1.25, 1.27 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: 30545ff1dee893be51df535b18cc1ac2c3bd6288
Maven: 0.23.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | --------------------- |
| core-lib-azure | 0.22.0 | 0.0.28 |
| core-lib-gc | 0.21.0-rc5 | |
| os-core-lib-aws | 0.21.0-rc5 | 0.21.0-rc5 |
| obm | 0.21.0-rc2 | |
| oqm | 0.21.0-rc4 | |
| os-core-common | 0.19.0-rc8, 0.21.0-rc5 | 0.13.0, 0.20.1 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.8.0 |
| osm | 0.21.0-rc3 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.5, 2.13.2.2, 2.13.4.2 | 2.13.2.2, 2.10.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.3.28, 5.3.24, 5.3.22 | 5.2.2.RELEASE, 5.3.12 |
| (3rd Party) org.yaml.snakeyaml | 2.0 | 1.25, 1.27 |M19 - Release 0.22https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/456Syncing AWS Code Commit to Gitlab2023-07-07T00:40:29ZSolomon AyalewSyncing AWS Code Commit to GitlabSyncing AWS Code Commit to Gitlab update versionSyncing AWS Code Commit to Gitlab update versionM19 - Release 0.22Solomon AyalewSolomon Ayalew