csv-parser merge requests
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests
2024-01-22T04:33:46Z
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/438
[ MSCOSDU-1955] remove git sha from dag name
2024-01-22T04:33:46Z
saketh somaraju
[ MSCOSDU-1955] remove git sha from dag name
**Current behavior**
- CSV parser dag name is appended with Git sha of commit which lead to lot of duplicate dags in GLAB env.
**Expected behavior**
- This MR removes appended git sha from dag name, dag file name and in the docker image...
**Current behavior**
- CSV parser dag name is appended with Git sha of commit which lead to lot of duplicate dags in GLAB env.
**Expected behavior**
- This MR removes appended git sha from dag name, dag file name and in the docker image.
- This would help in standardizing dags workflows
Changes are tested with https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/jobs/2485855#L293
M23 - Release 0.26
saketh somaraju
saketh somaraju
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/419
DRAFT: check oidc in gc jobs
2023-10-12T20:40:23Z
Aliaksandr Ramanovich (EPAM)
DRAFT: check oidc in gc jobs
check oidc in gc jobs
check oidc in gc jobs
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/415
Draft: fix azure jackson-databind vulnerability
2023-10-05T05:38:37Z
VidyaDharani Lokam
Draft: fix azure jackson-databind vulnerability
# Change details
* upgrade `jackson-databind` to `2.15.2`
# Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBM
# Change details
* upgrade `jackson-databind` to `2.15.2`
# Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBM
M21 - Release 0.24
VidyaDharani Lokam
VidyaDharani Lokam
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/411
Draft: Update Azure CI variables
2023-09-21T04:57:50Z
saketh somaraju
Draft: Update Azure CI variables
Made changes to Azure CI variables to accommodate dev secondary environment
Made changes to Azure CI variables to accommodate dev secondary environment
M21 - Release 0.24
saketh somaraju
saketh somaraju
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/400
Draft: dummy mr to test az copy dag stage
2023-07-31T10:37:20Z
saketh somaraju
Draft: dummy mr to test az copy dag stage
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/394
Upgrade First Party Library Dependencies for Release 0.22
2023-07-18T07:07:37Z
Chad Leong
Upgrade First Party Library Dependencies for Release 0.22
This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...
This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: e17ceb5fd857be7889ab92f5579f37448a1a68be
Maven: 0.23.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------ | ------------------ | -------------------------- |
| core-lib-azure | 0.15.0-rc4 | 0.15.0-rc4 |
| core-lib-gc | 0.21.0 | 0.21.0 |
| core-lib-gcp | | 0.19.0-rc3 |
| os-core-lib-aws | 0.21.0 | 0.21.0 |
| obm | 0.21.0 | 0.21.0 |
| oqm | 0.21.0 | 0.21.0 |
| os-core-common | 0.16.0-rc2, 0.21.0 | 0.13.0, 0.16.0-rc2, 0.17.0 |
| os-core-lib-ibm | 0.15.2 | 0.13.0 |
| osm | | 0.18.0 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade-2
SHA: 3194cb598b25548a02c005b6765236f797074a6d
Maven: 0.23.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------ | ------------------ | -------------------------- |
| core-lib-azure | 0.15.0-rc4 | 0.15.0-rc4 |
| core-lib-gc | 0.22.1 | 0.19.0-rc3 |
| os-core-lib-aws | 0.22.0 | 0.22.0 |
| obm | 0.22.0 | 0.22.0 |
| oqm | 0.22.0 | 0.22.0 |
| os-core-common | 0.16.0-rc2, 0.22.0 | 0.13.0, 0.16.0-rc2, 0.17.0 |
| os-core-lib-ibm | 0.15.2 | 0.13.0 |
M19 - Release 0.22
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/389
Draft: dummy MR
2023-08-07T06:01:40Z
Deepa Kumari
Draft: dummy MR
Deepa Kumari
Deepa Kumari
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/384
gc_fix_move_to_airflow2.6
2023-06-09T13:15:44Z
Yan Sushchynski (EPAM)
gc_fix_move_to_airflow2.6
Yan Sushchynski (EPAM)
Yan Sushchynski (EPAM)
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/363
Draft: GONRG-6974: Test DAG sync pipeline
2023-05-11T07:55:26Z
Volodymyr Pienskoi [EPAM / GCP]
Draft: GONRG-6974: Test DAG sync pipeline
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/360
Set FailOnError to false
2023-04-28T19:56:30Z
Yash Dholakia
Set FailOnError to false
Set FailOnError to false so build does not fail when dependency-check-maven fails.
Set FailOnError to false so build does not fail when dependency-check-maven fails.
Yash Dholakia
Yash Dholakia
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/357
fix libs conflicts
2023-10-11T16:15:22Z
Rustam Lotsmanenko (EPAM)
rustam_lotsmanenko@epam.com
fix libs conflicts
Rustam Lotsmanenko (EPAM)
rustam_lotsmanenko@epam.com
Rustam Lotsmanenko (EPAM)
rustam_lotsmanenko@epam.com
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/353
Full Upgrade of First Party Library Dependencies for Release 0.20
2023-05-22T15:03:46Z
David Diederich
d.diederich@opengroup.org
Full Upgrade of First Party Library Dependencies for Release 0.20
This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will ...
This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will often fail, since the upgrades were previously rejected for failing pipelines and have not been directly addressed yet.
This upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
This MR may co-exist with a separate, smaller upgrade MR.
If both pass, this one should be used instead.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 663e9bbd66cf55f715ab625fba7c0f092d0793a2
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | -------------------------- |
| core-lib-azure | 0.15.0-rc4 | 0.15.0-rc4 |
| core-lib-gcp | 0.19.0 | 0.19.0-rc3 |
| os-core-lib-aws | 0.14.0-rc2 | 0.14.0-rc2 |
| obm | 0.19.0 | 0.18.0 |
| oqm | 0.19.0 | 0.18.0 |
| os-core-common | 0.13.0, 0.15.0-rc4 | 0.13.0, 0.15.0-rc4, 0.17.0 |
| os-core-lib-ibm | 0.15.2 | 0.13.0 |
| osm | 0.19.0 | 0.18.0 |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.1, 2.10.5, 2.13.2.2 | 2.13.2.2, 2.10.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.3.22, 5.3.23 | 5.2.8.RELEASE |
| (3rd Party) org.yaml.snakeyaml | 1.33 | 1.33 |
```
Critical: Found Vulnerable Snake YAML dependency (<2.0)
├─ _Root_
│ ├─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.csv-parser-azure == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.core-lib-azure == 0.15.0-rc4
│ │ └─ org.redisson.redisson == 3.15.3
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.csv-parser-gc == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.csv-parser-ibm == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.33
│ └─ org.opengroup.osdu.csv-parser-aws == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
└─ testing/
├─ org.opengroup.osdu.csv-parser-core-test == 0.21.0-SNAPSHOT
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-azure-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-azure == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core-lib-azure == 0.15.0-rc4
│ └─ org.redisson.redisson == 3.15.3
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-aws-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-gc-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.17.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-ibm-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-lib-ibm == 0.13.0
│ └─ org.springframework.boot.spring-boot-starter-security == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
└─ org.opengroup.osdu.csv-parser-anthos-test == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.17.0
└─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
└─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
└─ org.yaml.snakeyaml == 1.33
```
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
├─ _Root_
│ └─ org.opengroup.osdu.csv-parser-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
└─ testing/
├─ org.opengroup.osdu.csv-parser-azure-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-azure == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-gc-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.17.0
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-ibm-test == 0.21.0-SNAPSHOT
│ └─ com.amazonaws.aws-java-sdk-core == 1.11.1018
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
└─ org.opengroup.osdu.csv-parser-anthos-test == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.17.0
└─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
```
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: 51be6b0abdc6f6c9885a32173da18a7de739dda2
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | ---------------- |
| core-lib-azure | 0.20.0 | 0.20.0 |
| core-lib-gc | 0.20.0 | 0.20.0 |
| os-core-lib-aws | 0.20.0 | 0.20.0 |
| obm | 0.20.0 | 0.20.0 |
| oqm | 0.20.0 | 0.20.0 |
| os-core-common | 0.20.1 | 0.20.1 |
| os-core-lib-ibm | 0.20.0 | 0.20.0 |
| osm | 0.20.0 | 0.20.0 |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.1, 2.10.5, 2.13.2.2 | 2.13.2.2, 2.10.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.3.22, 5.3.23 | 5.2.8.RELEASE |
| (3rd Party) org.yaml.snakeyaml | 1.33 | 1.33 |
```
Critical: Found Vulnerable Snake YAML dependency (<2.0)
├─ _Root_
│ ├─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.csv-parser-azure == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.core-lib-azure == 0.20.0
│ │ └─ org.redisson.redisson == 3.15.3
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.csv-parser-gc == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ │ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.csv-parser-ibm == 0.21.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.os-core-lib-ibm == 0.20.0
│ │ └─ org.yaml.snakeyaml == 1.33
│ └─ org.opengroup.osdu.csv-parser-aws == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.11.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.11.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
└─ testing/
├─ org.opengroup.osdu.csv-parser-core-test == 0.21.0-SNAPSHOT
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-azure-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-azure == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core-lib-azure == 0.20.0
│ └─ org.redisson.redisson == 3.15.3
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-aws-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.core.aws.os-core-lib-aws == 0.20.0
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-gc-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
│ └─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
│ └─ org.yaml.snakeyaml == 1.33
├─ org.opengroup.osdu.csv-parser-ibm-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-lib-ibm == 0.20.0
│ └─ org.yaml.snakeyaml == 1.33
└─ org.opengroup.osdu.csv-parser-anthos-test == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.20.1
└─ org.springframework.boot.spring-boot-starter-web == 2.2.5.RELEASE
└─ org.springframework.boot.spring-boot-starter == 2.2.5.RELEASE
└─ org.yaml.snakeyaml == 1.33
```
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
├─ _Root_
│ └─ org.opengroup.osdu.csv-parser-gc == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-core == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
└─ testing/
├─ org.opengroup.osdu.csv-parser-azure-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-azure == 0.21.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-gc-test == 0.21.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.20.1
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-ibm-test == 0.21.0-SNAPSHOT
│ └─ com.amazonaws.aws-java-sdk-core == 1.11.1018
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
└─ org.opengroup.osdu.csv-parser-anthos-test == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.20.1
└─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
```
M18 - Release 0.21
Okoun-Ola Fabien Houeto
Brindaban Das
Yauhen Shaliou [EPAM/GCP]
Srinivasan Narayanan
Okoun-Ola Fabien Houeto
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/350
Upgrade to SnakeYaml 2.0
2023-03-31T17:32:40Z
Abhay Joshi
Upgrade to SnakeYaml 2.0
commit b7a2aac1
Author: Abhay <bios@amazon.com>
Date: Fri Dec 02 2022 10:37:59 GMT-0800 (Pacific Standard Time)
upgrade dependencies
commit b7a2aac1
Author: Abhay <bios@amazon.com>
Date: Fri Dec 02 2022 10:37:59 GMT-0800 (Pacific Standard Time)
upgrade dependencies
M17 - Release 0.20
Okoun-Ola Fabien Houeto
Abhay Joshi
Okoun-Ola Fabien Houeto
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/348
Draft: GONRG-5597-disable csps, use custom branch
2023-06-06T13:02:25Z
Aliaksandr Ramanovich (EPAM)
Draft: GONRG-5597-disable csps, use custom branch
Test MR
Test MR
Aliaksandr Ramanovich (EPAM)
Aliaksandr Ramanovich (EPAM)
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/341
Upgrade First Party Library Dependencies for Release 0.19
2023-02-18T07:29:31Z
David Diederich
d.diederich@opengroup.org
Upgrade First Party Library Dependencies for Release 0.19
This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...
This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 7a3c54c576cd29b319b6f531f6a1254bb50e46c1
Maven: 0.20.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | -------------------------- |
| core-lib-azure | 0.15.0-rc4 | 0.15.0-rc4 |
| core-lib-gcp | 0.16.0-rc1 | 0.19.0-rc3 |
| os-core-lib-aws | 0.14.0-rc2 | 0.14.0-rc2 |
| obm | 0.15.0 | 0.18.0 |
| oqm | 0.15.0 | 0.18.0 |
| os-core-common | 0.13.0, 0.15.0-rc4 | 0.13.0, 0.15.0-rc4, 0.17.0 |
| os-core-lib-ibm | 0.15.2 | 0.13.0 |
| osm | 0.15.0 | 0.18.0 |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.1, 2.10.5, 2.13.2.2 | 2.13.2.2, 2.10.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.3.22 | 5.2.8.RELEASE |
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
├─ _Root_
│ ├─ org.opengroup.osdu.csv-parser-gc == 0.20.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.csv-parser-core == 0.20.0-SNAPSHOT
│ │ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
│ └─ org.opengroup.osdu.csv-parser-ibm == 0.20.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-core == 0.20.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
└─ testing/
├─ org.opengroup.osdu.csv-parser-azure-test == 0.20.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-azure == 0.20.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-gc-test == 0.20.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.17.0
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-ibm-test == 0.20.0-SNAPSHOT
│ └─ com.amazonaws.aws-java-sdk-core == 1.11.1018
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
└─ org.opengroup.osdu.csv-parser-anthos-test == 0.20.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.17.0
└─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
```
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: 7f1368b7ba5e4c664ca8e4b9e5d7389c2dbd2af4
Maven: 0.20.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | -------------------------- |
| core-lib-azure | 0.15.0-rc4 | 0.15.0-rc4 |
| core-lib-gcp | 0.16.0-rc1 | 0.19.0 |
| os-core-lib-aws | 0.14.0-rc2 | 0.14.0-rc2 |
| obm | 0.15.0 | 0.19.0 |
| oqm | 0.15.0 | 0.19.0 |
| os-core-common | 0.13.0, 0.15.0-rc4 | 0.13.0, 0.15.0-rc4, 0.17.0 |
| os-core-lib-ibm | 0.15.2 | 0.13.0 |
| osm | 0.15.0 | 0.19.0 |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.1, 2.10.5, 2.13.2.2 | 2.13.2.2, 2.10.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.12.1 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.12.1 |
| (3rd Party) org.springframework.spring-webmvc | 5.3.22 | 5.2.8.RELEASE |
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
├─ _Root_
│ ├─ org.opengroup.osdu.csv-parser-gc == 0.20.0-SNAPSHOT
│ │ └─ org.opengroup.osdu.csv-parser-core == 0.20.0-SNAPSHOT
│ │ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
│ └─ org.opengroup.osdu.csv-parser-ibm == 0.20.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-core == 0.20.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.5
└─ testing/
├─ org.opengroup.osdu.csv-parser-azure-test == 0.20.0-SNAPSHOT
│ └─ org.opengroup.osdu.csv-parser-azure == 0.20.0-SNAPSHOT
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-gc-test == 0.20.0-SNAPSHOT
│ └─ org.opengroup.osdu.os-core-common == 0.17.0
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
├─ org.opengroup.osdu.csv-parser-ibm-test == 0.20.0-SNAPSHOT
│ └─ com.amazonaws.aws-java-sdk-core == 1.11.1018
│ └─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
└─ org.opengroup.osdu.csv-parser-anthos-test == 0.20.0-SNAPSHOT
└─ org.opengroup.osdu.os-core-common == 0.17.0
└─ com.fasterxml.jackson.core.jackson-databind == 2.10.2
```
M16 - Release 0.19
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/335
Trusted users/ss/csv parser retries master
2023-08-25T22:21:53Z
Srishti Sharma
Trusted users/ss/csv parser retries master
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/334
Trusted users/ss/csv parser retries master
2023-01-04T12:14:50Z
Srishti Sharma
Trusted users/ss/csv parser retries master
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/333
Trusted users/ss/csv parser retries master
2023-01-03T05:14:20Z
Srishti Sharma
Trusted users/ss/csv parser retries master
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/332
Configure retries for CSV Parser in Master
2023-08-25T22:21:51Z
Srishti Sharma
Configure retries for CSV Parser in Master
Configured Operator level retries
Tested the retries on a test instance and verified the functionality is working.
Pipeline link: https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/pipelines/1570...
Configured Operator level retries
Tested the retries on a test instance and verified the functionality is working.
Pipeline link: https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/pipelines/157082
Trusted pipeline: https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/334/pipelines
https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/325
Configured retries for CSV Parser
2022-12-17T11:03:02Z
Srishti Sharma
Configured retries for CSV Parser
**PBI:** https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_workitems/edit/15576
Setting up retries for CSV Parser Dag
==============
Tested the retries on a test instance and verified the functionality is working.
Test ...
**PBI:** https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_workitems/edit/15576
Setting up retries for CSV Parser Dag
==============
Tested the retries on a test instance and verified the functionality is working.
Test Dags:
- https://adf-4q2m464plpuwq-airflowir-2707.westeurope.airflow.svc.datafactory.azure.com/tree?dag_id=sstest_Osdu_ingest
- https://adf-4q2m464plpuwq-airflowir-2707.westeurope.airflow.svc.datafactory.azure.com/tree?dag_id=sstest_csv-parser
harshit aggarwal
harshit aggarwal