Ingestion Workflow merge requestshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests2023-08-18T11:18:27Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/165Disable accepting dag content based on feature flag [azure]2023-08-18T11:18:27ZAalekh JainDisable accepting dag content based on feature flag [azure]M10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/193Update .gitlab-ci.yml2023-08-18T11:18:06ZAnuj GuptaUpdate .gitlab-ci.ymlM10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/194Airflow2 stable api support2023-08-18T11:18:04ZSaurabh TripathiAirflow2 stable api support### Summary of changes
* Make changes to the connect to airflow2 stable API and accommodate the response changes
* UT for the airflow2 implementations
### Related changes
* add airflow env selection variable in pipeline
* Documentation### Summary of changes
* Make changes to the connect to airflow2 stable API and accommodate the response changes
* UT for the airflow2 implementations
### Related changes
* add airflow env selection variable in pipeline
* DocumentationM10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/197[Azure] Add support for caching in workflow ingestion service2023-08-18T11:18:02ZAalekh Jain[Azure] Add support for caching in workflow ingestion serviceM10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/199[Azure] Limiting trigger workflow requests in workflow ingestion service2023-08-18T11:18:00ZAalekh Jain[Azure] Limiting trigger workflow requests in workflow ingestion serviceM10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/200Stabilize Workflow with postman collection (new Approach) (GONRG-3729)2023-08-18T11:17:59ZArtem Dobrynin (EPAM)Stabilize Workflow with postman collection (new Approach) (GONRG-3729)# Description:
There was an issue while creating system workflow entities. The flag has been ignored, therefore, some logic for system workflow has been skipping.
Also, there was some miswritten exception for workflow not found
# How t...# Description:
There was an issue while creating system workflow entities. The flag has been ignored, therefore, some logic for system workflow has been skipping.
Also, there was some miswritten exception for workflow not found
# How to test:
Functionality was tested on GCP with datastore and postgres
# Changes include:
* [ ] Refactor (a non-breaking change that improves code maintainability).
* [X] Bugfix (a non-breaking change that solves an issue).
* [ ] New feature (a non-breaking change that adds functionality).
* [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
* [x] GCP
* [ ] Azure
* [ ] AWS
* [ ] IBM
* [x] Common code
# Dev Checklist:
* [x] Added Unit Tests, wherever applicable.
* [x] Updated the Readme, if applicable.
* [x] Existing Tests pass
* [x] Verified functionality locally
* [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:M10 - Release 0.13Riabokon Stanislav(EPAM)[GCP]Rostislav Dublin (EPAM)Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/201Disable spring web security 2052023-08-18T11:17:57ZBhushan RadeDisable spring web security 205M10 - Release 0.13Anuj GuptaBhushan RadeAnuj Guptahttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/207Vuln fix lucene2023-08-18T11:17:55ZGokul NagareVuln fix luceneM10 - Release 0.13Anuj GuptaShrikant GargAnuj Guptahttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/208log4j vuln fix2023-08-18T11:17:54ZBhushan Radelog4j vuln fix|module pom changes| Ref Issue|
|------|-----------------------------------------------------------------------------------------------|
| IBM | https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm/-/issues...|module pom changes| Ref Issue|
|------|-----------------------------------------------------------------------------------------------|
| IBM | https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm/-/issues/2 |
| Core | https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/54 |
Part of https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/134 seriesM10 - Release 0.13David Diederichd.diederich@opengroup.orgAnuj GuptaBhushan RadeShrikant GargDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/209Disables unsafe subst for log4j2023-08-18T11:17:52ZOleksii Tsyganov (EPAM)Disables unsafe subst for log4jM10 - Release 0.13Oleksii Tsyganov (EPAM)Oleksii Tsyganov (EPAM)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/213Code coverage2023-08-18T11:17:50ZAalekh JainCode coverageTasks accomplished as part of this MR
1. Added the support for generating code coverage reports using `Jacoco`
2. Updated the settings for CI/CD pipelines (Under section General Pipelines) to capture the code coverage using the Test Cov...Tasks accomplished as part of this MR
1. Added the support for generating code coverage reports using `Jacoco`
2. Updated the settings for CI/CD pipelines (Under section General Pipelines) to capture the code coverage using the Test Coverage Parsing regex as: `Total.*?([0-9]{1,3})%`
3. Added code coverage badge for azure in the `README.md` corresponding to azure cloud provider.
Attached below are the screenshots of the code coverage captured through bade and in the pipeline stages -
1. ![image](/uploads/4a53a8708293ed392a535e2d0186f455/image.png)
2. ![image](/uploads/4e989b4ccad8ab35203cfb634a70b2ed/image.png)M10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/214Upgrading library versions to incorporate the latest Log4j fix2023-08-18T11:17:49ZDavid Diederichd.diederich@opengroup.orgUpgrading library versions to incorporate the latest Log4j fixThese updates apply version 2.17, addressing CVE-2021-45105
Part of the #135 seriesThese updates apply version 2.17, addressing CVE-2021-45105
Part of the #135 seriesM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/217Azure deploy fixes2023-08-18T11:17:47ZAalekh JainAzure deploy fixesM10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/219Airflow2 upgrade doc2023-08-18T11:17:45ZSaurabh TripathiAirflow2 upgrade docM10 - Release 0.13Vineeth Guna [Microsoft]Vineeth Guna [Microsoft]https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/220Ibm core lib upgrade2023-08-18T11:17:44ZAshwani PandeyIbm core lib upgradeIBM core lib version upgrade to rc6IBM core lib version upgrade to rc6M10 - Release 0.13Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/223Update GKE SA to follow common naming convention [GONRG-4143]2023-08-18T11:17:42ZMikhail Piatliou (EPAM)Update GKE SA to follow common naming convention [GONRG-4143]M10 - Release 0.13Mikhail Piatliou (EPAM)Mikhail Piatliou (EPAM)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/224Revert "Merge branch 'gcp-update-gke-sa' into 'master'"2023-08-18T11:17:41ZOleksandr Kosse (EPAM)Revert "Merge branch 'gcp-update-gke-sa' into 'master'"This reverts merge request !223This reverts merge request !223M10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/225Update GKE SA to follow common naming convention [GONRG-4143]2023-08-18T11:17:39ZMikhail Piatliou (EPAM)Update GKE SA to follow common naming convention [GONRG-4143]M10 - Release 0.13Mikhail Piatliou (EPAM)Mikhail Piatliou (EPAM)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/226GONRG-4118 Add imagePullPolicy value2023-08-18T11:17:37ZIgor Zimovets (EPAM)GONRG-4118 Add imagePullPolicy valueM10 - Release 0.13https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/227GONRG-4206: Verify dev2 pipeline for workflow2023-08-18T11:17:36ZNastassia Rabeichykava (EPAM)GONRG-4206: Verify dev2 pipeline for workflowM10 - Release 0.13Oleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)