Ingestion Workflow merge requestshttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests2023-08-18T13:06:20Zhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/462OpenAPI 3.0 Documentation using springdoc2023-08-18T13:06:20ZJayesh BagulOpenAPI 3.0 Documentation using springdoc**Link to ADR(Architecture Decision Record)** : [Swagger using springdoc-openapi](https://community.opengroup.org/osdu/platform/system/home/-/issues/97)
## OpenAPI 3.0 related changes
* upgraded to latest **springdoc openapi** latest v...**Link to ADR(Architecture Decision Record)** : [Swagger using springdoc-openapi](https://community.opengroup.org/osdu/platform/system/home/-/issues/97)
## OpenAPI 3.0 related changes
* upgraded to latest **springdoc openapi** latest version [1.7.0](https://mvnrepository.com/artifact/org.springdoc/springdoc-openapi-ui/1.6.14)
* Documented the below API's with OpenAPI 3.0 **Annotations**
- WorkflowManager API
- WorkflowRun API
- WorkflowSystemManager API
- HealthCheck API
- Info API
* Added the standard HTTP Response(4xx, 5x\*\*\*\*x) for API Responses
* Custom Path for
* **Swagger UI**: https://host/context-path/swagger (will redirect to https://host/context-path/swagger-ui/index.html)
* **api-docs (JSON)** : https://host/context-path/api-docs
* **api-docs (YAML)** : https://host/context-path/api-docs.yaml
* Azure Swagger GLAB(for Reference)
* **Swagger UI**: https://osdu-glab.msft-osdu-test.org/api/workflow/swagger (will redirect to https://osdu-glab.msft-osdu-test.org/api/workflow/swagger-ui/index.html)
* **api-docs (JSON)** : https://osdu-glab.msft-osdu-test.org/api/workflow/api-docs
* **api-docs (YAML)** :https://osdu-glab.msft-osdu-test.org/api/workflow/api-docs.yaml
* Marked the below **Internal** API's as **Hidden**
- Azure WhoamiController
- Azure CustomOperatorApi
## Other Changes
- **Configurable** descriptions managed in [swagger.properties](https://community.opengroup.org/osdu/platform/system/search/-/blob/az/td-oas/search-core/src/main/resources/swagger.properties)
- Deleted [HomeController, HomeControllerTest, SpringfoxSwaggerHostResolver, SpringfoxSwaggerHostResolverTest,
SwaggerDocumentationConfig]
- Updated Readme for swagger related information
## References
- https://springdoc.org/faq.html#_can_i_use_spring_property_with_swagger_annotations
- https://springdoc.org/migrating-from-springfox.htmlM20 - Release 0.23Jayesh BagulJayesh Bagulhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/461Cherry-pick 'Adding provider specific artifact versions' into release/0.222023-07-28T10:40:03ZChad LeongCherry-pick 'Adding provider specific artifact versions' into release/0.22**Original MR**: !460
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !460
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/pipelines/new?ref=cherry-pick-for-460)M19 - Release 0.22David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/460Adding provider specific artifact versions2023-07-28T10:23:48ZDavid Diederichd.diederich@opengroup.orgAdding provider specific artifact versionsThis enables provider specific releases, incrementing the version of only one componentThis enables provider specific releases, incrementing the version of only one componentM19 - Release 0.22David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/457Cherry-pick '[Dependency Updates] WhiteSource fixes [jul2023]' into release/0.222023-07-13T13:23:00ZChad LeongCherry-pick '[Dependency Updates] WhiteSource fixes [jul2023]' into release/0.22**Original MR**: !455
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !455
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/pipelines/new?ref=cherry-pick-for-455)M19 - Release 0.22David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/455[Dependency Updates] WhiteSource fixes [jul2023]2023-07-13T09:26:49ZMaksim Malkov[Dependency Updates] WhiteSource fixes [jul2023]### Version updates
[root-pom]
* json-smart -> 2.4.11
* netty-all -> 4.1.72.Final
* netty-handler -> 4.1.72.Final
* netty-codec -> 4.1.68.Final
* netty-common -> 4.1.71.Final
* jersey-client -> 1.19.4
* jackson-core -> 2.13.2
* jackson-...### Version updates
[root-pom]
* json-smart -> 2.4.11
* netty-all -> 4.1.72.Final
* netty-handler -> 4.1.72.Final
* netty-codec -> 4.1.68.Final
* netty-common -> 4.1.71.Final
* jersey-client -> 1.19.4
* jackson-core -> 2.13.2
* jackson-annotations -> 2.13.2
* jackson-databind -> 2.13.5
* guava -> 32.1.1-jre
* snakeyaml -> 2.0
* spring-security-web -> 5.7.8
* tomcat-embed-core -> 9.0.76
* spring-beans -> 5.3.28
* spring-core -> 5.3.28
* spring-context -> 5.3.28
* spring-expression -> 5.3.28
* spring-web -> 5.3.28
* spring-webmvc -> 5.3.28
* tomcat-coyote -> 9.0.76
* json -> 20230618
[azure]
* tomcat-coyote -> 9.0.76M19 - Release 0.22Maksim MalkovMaksim Malkovhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/448Updated maven dependency2023-06-19T08:25:15ZShreya ShahUpdated maven dependency# Fixes :
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/security/vulnerabilities/20043
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/security/vulnerabiliti...# Fixes :
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/security/vulnerabilities/20043
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/security/vulnerabilities/20040
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/security/vulnerabilities/21542
https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/security/vulnerabilities/21539
# Issue :
https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/167
https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/166M19 - Release 0.22Shreya ShahShreya Shahhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/446Cherry-pick 'Bugfix initialize executionContext' into release/0.212023-06-13T19:12:41ZChad LeongCherry-pick 'Bugfix initialize executionContext' into release/0.21**Original MR**: !444
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !444
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/pipelines/new?ref=cherry-pick-for-444)M18 - Release 0.21David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/444Bugfix initialize executionContext2023-06-13T19:12:24ZMadalyn MarabellaBugfix initialize executionContextThe `addUserId` method is throwing an error on `executionContext.get(KEY_USER_ID)`. Initializing executionContext fixes the problem. I would like input on whether this is the best place to initialize.The `addUserId` method is throwing an error on `executionContext.get(KEY_USER_ID)`. Initializing executionContext fixes the problem. I would like input on whether this is the best place to initialize.M18 - Release 0.21Madalyn MarabellaMadalyn Marabellahttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/440Cherrypick dependency vulnerability upgrades2023-06-08T04:25:15ZMadalyn MarabellaCherrypick dependency vulnerability upgradesOriginal MR https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/413Original MR https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/413M18 - Release 0.21Madalyn MarabellaMadalyn Marabellahttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/436Cherry-pick 'Pass 'user id' as part of Airflow config' into release/0.212023-05-29T09:11:21ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Pass 'user id' as part of Airflow config' into release/0.21**Original MR**: !407
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !407
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/pipelines/new?ref=cherry-pick-for-407)M18 - Release 0.21David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/413Upgrade vulnerable dependencies2023-06-05T09:54:40ZYash DholakiaUpgrade vulnerable dependencies-Upgrade Sping-boot-starter-web to 2.7.10
-Upgrade SnakeYaml to 2.0-Upgrade Sping-boot-starter-web to 2.7.10
-Upgrade SnakeYaml to 2.0M18 - Release 0.21Yash DholakiaYash Dholakiahttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/407Pass 'user id' as part of Airflow config2023-05-26T16:53:29ZThulasi Dass SubramanianPass 'user id' as part of Airflow configRelated ADR - https://community.opengroup.org/osdu/platform/data-flow/ingestion/home/-/issues/52
In accordance with the above ADR change involves adding the 'userId' field into execution context in Airflow Config ObjectRelated ADR - https://community.opengroup.org/osdu/platform/data-flow/ingestion/home/-/issues/52
In accordance with the above ADR change involves adding the 'userId' field into execution context in Airflow Config ObjectM18 - Release 0.21Thulasi Dass SubramanianDeepa KumariThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/402Cherry-pick 'Upgrade Dependencies' into release/0.202023-04-11T09:48:06ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Upgrade Dependencies' into release/0.20**Original MR**: !395
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !395
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/pipelines/new?ref=cherry-pick-for-395)M17 - Release 0.20David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/401Full Upgrade of First Party Library Dependencies for Release 0.202023-05-22T15:17:48ZDavid Diederichd.diederich@opengroup.orgFull Upgrade of First Party Library Dependencies for Release 0.20This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will ...This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will often fail, since the upgrades were previously rejected for failing pipelines and have not been directly addressed yet.
This upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
This MR may co-exist with a separate, smaller upgrade MR.
If both pass, this one should be used instead.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: ad4e2951c9d4004fb28604322f95c742906dbbb8
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | ---------------------------------------------- | --------------------- |
| core-lib-azure | 0.14.0-rc2 | 0.0.28 |
| core-lib-gcp | 0.19.0-rc3 | |
| os-core-lib-aws | 0.19.0-rc3 | 0.14.0-rc2 |
| obm | 0.18.0 | |
| oqm | 0.18.0 | |
| os-core-common | 0.19.0-rc8 | 0.13.0 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.8.0 |
| osm | 0.18.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.10.5, 2.13.2.2, 2.13.4.2 | 2.13.2.2, 2.10.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.2.10.RELEASE, 5.2.22.RELEASE, 5.3.24, 5.3.22 | 5.2.2.RELEASE, 5.3.12 |
| (3rd Party) org.yaml.snakeyaml | 1.25, 2.0 | 1.25, 1.27 |
```
Critical: Found Vulnerable Spring MVC dependency (<5.2.20 || >=5.3.0 <5.3.18)
├─ _Root_
│ ├─ io.springfox.springfox-boot-starter == 3.0.0
│ │ └─ org.opengroup.osdu.os-core-common == 0.19.0-rc8
│ │ └─ org.springframework.spring-webmvc == 5.2.10.RELEASE
│ └─ org.opengroup.osdu.workflow-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-core == 0.21.0-SNAPSHOT
│ └─ org.springframework.spring-webmvc == 5.2.10.RELEASE
└─ testing/
├─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
├─ org.opengroup.osdu.workflow.workflow-test-aws == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core.aws.os-core-lib-aws == 0.14.0-rc2
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
├─ org.opengroup.osdu.workflow-test-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
├─ org.opengroup.osdu.workflow.workflow-test-azure == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.4.12
│ └─ org.springframework.spring-webmvc == 5.3.12
├─ org.opengroup.osdu.workflow-test-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
└─ org.opengroup.osdu.workflow-test-anthos == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.13.0
└─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
└─ org.springframework.spring-webmvc == 5.2.2.RELEASE
```
```
Critical: Found Vulnerable Snake YAML dependency (<2.0)
├─ _Root_
│ ├─ io.springfox.springfox-boot-starter == 3.0.0
│ │ └─ org.springframework.cloud.spring-cloud-starter == 2.2.2.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.25
│ ├─ org.opengroup.osdu.workflow-aws == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.workflow-core == 0.21.0-SNAPSHOT
│ │ └─ org.yaml.snakeyaml == 1.25
│ └─ org.opengroup.osdu.workflow-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-core == 0.21.0-SNAPSHOT
│ └─ org.yaml.snakeyaml == 1.25
└─ testing/
├─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
├─ org.opengroup.osdu.workflow.workflow-test-aws == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core.aws.os-core-lib-aws == 0.14.0-rc2
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
├─ org.opengroup.osdu.workflow-test-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
├─ org.opengroup.osdu.workflow.workflow-test-azure == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.4.12
│ └─ org.springframework.boot.spring-boot-starter == 2.4.12
│ └─ org.yaml.snakeyaml == 1.27
├─ org.opengroup.osdu.workflow-test-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-lib-ibm == 0.8.0
│ └─ org.springframework.boot.spring-boot-starter-security == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
└─ org.opengroup.osdu.workflow-test-anthos == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.13.0
└─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
└─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
└─ org.yaml.snakeyaml == 1.25
```
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
├─ _Root_
│ └─ io.springfox.springfox-boot-starter == 3.0.0
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
└─ testing/
├─ org.opengroup.osdu.workflow-test-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
├─ org.opengroup.osdu.workflow.workflow-test-azure == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
├─ org.opengroup.osdu.workflow-test-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
└─ org.opengroup.osdu.workflow-test-anthos == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
└─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
```
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: da526b77c99d93cb37e06a34420722ff6652514a
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | ---------------------------------------------- | --------------------- |
| core-lib-azure | 0.20.0 | 0.20.0 |
| core-lib-gc | 0.20.0 | |
| os-core-lib-aws | 0.20.0 | 0.20.0 |
| obm | 0.20.0 | |
| oqm | 0.20.0 | |
| os-core-common | 0.20.1 | 0.20.1 |
| os-core-lib-ibm | 0.20.0 | 0.20.0 |
| osm | 0.20.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.10.5, 2.13.2.2, 2.13.4.2 | 2.13.2.2, 2.10.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.2.10.RELEASE, 5.2.22.RELEASE, 5.3.24, 5.3.22 | 5.2.2.RELEASE, 5.3.22 |
| (3rd Party) org.yaml.snakeyaml | 1.25, 2.0 | 1.25, 1.27 |
```
Critical: Found Vulnerable Spring MVC dependency (<5.2.20 || >=5.3.0 <5.3.18)
├─ _Root_
│ ├─ io.springfox.springfox-boot-starter == 3.0.0
│ │ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ │ └─ org.springframework.spring-webmvc == 5.2.10.RELEASE
│ └─ org.opengroup.osdu.workflow-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-core == 0.21.0-SNAPSHOT
│ └─ org.springframework.spring-webmvc == 5.2.10.RELEASE
└─ testing/
├─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
├─ org.opengroup.osdu.workflow.workflow-test-aws == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core.aws.os-core-lib-aws == 0.20.0
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
├─ org.opengroup.osdu.workflow-test-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
├─ org.opengroup.osdu.workflow-test-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ org.springframework.spring-webmvc == 5.2.2.RELEASE
└─ org.opengroup.osdu.workflow-test-anthos == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.20.1
└─ org.springframework.spring-webmvc == 5.2.2.RELEASE
```
```
Critical: Found Vulnerable Snake YAML dependency (<2.0)
├─ _Root_
│ ├─ io.springfox.springfox-boot-starter == 3.0.0
│ │ └─ org.springframework.cloud.spring-cloud-starter == 2.2.2.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.25
│ ├─ org.opengroup.osdu.workflow-aws == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.workflow-core == 0.21.0-SNAPSHOT
│ │ └─ org.yaml.snakeyaml == 1.25
│ └─ org.opengroup.osdu.workflow-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-core == 0.21.0-SNAPSHOT
│ └─ org.yaml.snakeyaml == 1.25
└─ testing/
├─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
├─ org.opengroup.osdu.workflow.workflow-test-aws == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core.aws.os-core-lib-aws == 0.20.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
├─ org.opengroup.osdu.workflow-test-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
│ └─ org.yaml.snakeyaml == 1.25
├─ org.opengroup.osdu.workflow.workflow-test-azure == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core-lib-azure == 0.20.0
│ └─ org.redisson.redisson == 3.15.3
│ └─ org.yaml.snakeyaml == 1.27
├─ org.opengroup.osdu.workflow-test-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-lib-ibm == 0.20.0
│ └─ org.yaml.snakeyaml == 1.25
└─ org.opengroup.osdu.workflow-test-anthos == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.20.1
└─ org.springframework.boot.spring-boot-starter-web == 2.2.2.RELEASE
└─ org.springframework.boot.spring-boot-starter == 2.2.2.RELEASE
└─ org.yaml.snakeyaml == 1.25
```
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
├─ _Root_
│ └─ io.springfox.springfox-boot-starter == 3.0.0
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
└─ testing/
├─ org.opengroup.osdu.workflow-test-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
├─ org.opengroup.osdu.workflow.workflow-test-azure == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
├─ org.opengroup.osdu.workflow-test-ibm == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
└─ org.opengroup.osdu.workflow-test-anthos == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.workflow-test-core == 0.21.0-SNAPSHOT
└─ com.fasterxml.jackson.core.jackson-databind == 2.10.1
```M18 - Release 0.21David Diederichd.diederich@opengroup.orgSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/395Upgrade Dependencies2023-05-04T19:19:55ZAbhay JoshiUpgrade Dependenciescommit 50a58f7d
Author: Abhay <bios@amazon.com>
Date: Mon Mar 27 2023 11:00:10 GMT-0700 (Pacific Daylight Time)
upgrading dependencies
(cherry picked from commit 33772b30b0003ace37c1af14e557eb98761f0dfd)commit 50a58f7d
Author: Abhay <bios@amazon.com>
Date: Mon Mar 27 2023 11:00:10 GMT-0700 (Pacific Daylight Time)
upgrading dependencies
(cherry picked from commit 33772b30b0003ace37c1af14e557eb98761f0dfd)M17 - Release 0.20Okoun-Ola Fabien HouetoAbhay JoshiOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/378Update vulnerable lib versions2023-07-25T07:11:22ZManish JangidUpdate vulnerable lib versions* Updating versions of vulnerable libraries
* https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/123
* https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/124
* https...* Updating versions of vulnerable libraries
* https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/123
* https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/124
* https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/125
* https://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/130M16 - Release 0.19https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/368jar type vulnerability fix for ingestion-workflow-service2023-03-30T11:26:01ZPintu Guptajar type vulnerability fix for ingestion-workflow-serviceFollowing CVE has been fix in this MR :
| cve | link |
|------------------|-------------------------------------------------|
| CVE-2022-25857 | https://nvd.nist.gov/vuln/detail/...Following CVE has been fix in this MR :
| cve | link |
|------------------|-------------------------------------------------|
| CVE-2022-25857 | https://nvd.nist.gov/vuln/detail/CVE-2022-25857 |
| CVE-2021-22118 | https://nvd.nist.gov/vuln/detail/CVE-2021-22118 |
| PRISMA-2022-0239 | https://github.com/square/okhttp/issues/6738 |
| CVE-2022-42252 | https://nvd.nist.gov/vuln/detail/CVE-2022-42252 |
| CVE-2022-22965 | https://nvd.nist.gov/vuln/detail/CVE-2022-22965 |
| CVE-2022-25647 | https://nvd.nist.gov/vuln/detail/CVE-2022-25647 |
| CVE-2022-22965 | https://nvd.nist.gov/vuln/detail/CVE-2022-22965 |
| CVE-2022-31692 | https://nvd.nist.gov/vuln/detail/CVE-2022-31692 |
| CVE-2022-22965 | https://nvd.nist.gov/vuln/detail/CVE-2022-22965 |
| CVE-2021-22118 | https://nvd.nist.gov/vuln/detail/CVE-2021-22118 |
| CVE-2022-22968 | https://nvd.nist.gov/vuln/detail/CVE-2022-22968 |
| CVE-2022-22970 | https://nvd.nist.gov/vuln/detail/CVE-2022-22970 |
| CVE-2022-42003 | https://nvd.nist.gov/vuln/detail/CVE-2022-42003 |
| CVE-2022-42004 | https://nvd.nist.gov/vuln/detail/CVE-2022-42004 |M17 - Release 0.20Pintu GuptaPintu Guptahttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/363Bugfix: fix misleading logs2023-05-22T07:22:38ZMaksim MalkovBugfix: fix misleading logs## Changes
- change log level for messages which printed from repository level during metadata search in both system and partition DBs
- update searching metadata logic accordingly (we do not need exception based branching here)
- update...## Changes
- change log level for messages which printed from repository level during metadata search in both system and partition DBs
- update searching metadata logic accordingly (we do not need exception based branching here)
- update unit and component tests
[issue link](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/150)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/341Upgrade Tomcat2022-10-06T02:10:26ZXiangliang MengUpgrade Tomcatcommit 730fb00c
Author: David Meng <xlmeng@amazon.com>
Date: Wed Sep 28 2022 15:37:56 GMT-0400 (Eastern Daylight Time)
Upgrade Tomcatcommit 730fb00c
Author: David Meng <xlmeng@amazon.com>
Date: Wed Sep 28 2022 15:37:56 GMT-0400 (Eastern Daylight Time)
Upgrade TomcatM14 - Release 0.17Okoun-Ola Fabien HouetoXiangliang MengOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/325Cherry pick spring-webmvc to release/0.162022-08-10T19:02:35ZMorris EstepaCherry pick spring-webmvc to release/0.16spring-webmvc upgrade
See merge request osdu/platform/data-flow/ingestion/ingestion-workflow!322
(cherry picked from commit 6675161cd9f9e82b6bbbbe6db9f9687b53426708)
1265d6cc Replace printStackTrace() with logging
360a5e70 Merge branc...spring-webmvc upgrade
See merge request osdu/platform/data-flow/ingestion/ingestion-workflow!322
(cherry picked from commit 6675161cd9f9e82b6bbbbe6db9f9687b53426708)
1265d6cc Replace printStackTrace() with logging
360a5e70 Merge branch 'master' into master-dev-merge
1827637c Merge remote-tracking branch 'origin/master' into...
f994d91d Merge branch 'master' into master-dev-merge
05709273 Merge remote-tracking branch 'origin/master' into...
cfd66b6b Squashed commit of the following:
1d4827d5 Merge branch 'master' into master-dev-merge
75ca9035 Merge branch 'master' into master-dev-merge
b46a8f4a Merge branch 'master' into master-dev-merge-saccomi
ba770f3e Squashed commit of the following
a4dc87c9 fix buildM13 - Release 0.16Morris EstepaMorris Estepa