Indexer Queue merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests2022-02-23T14:32:49Zhttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/170add record latency metrics2022-02-23T14:32:49ZNeelesh Thakuradd record latency metricsUsers have reported Indexing delays when Indexer service gets bursty traffic. This change-set captures the Storage --> Indexer latency to provide meaningful/accurate average Indexing latency.Users have reported Indexing delays when Indexer service gets bursty traffic. This change-set captures the Storage --> Indexer latency to provide meaningful/accurate average Indexing latency.M11 - Release 0.14https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/172Fix gcp indexation for multiple records and reindexing.2022-02-28T14:30:22ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comFix gcp indexation for multiple records and reindexing.Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/179Merge branch 'anuj/og-master-jakson-databind-fix' into 'master'2022-03-24T05:38:26ZAnuj GuptaMerge branch 'anuj/og-master-jakson-databind-fix' into 'master'Changes for vulnerability remediation
See merge request osdu/platform/system/indexer-queue!178
(cherry picked from commit b6ae94c5481cc5395fb86aae920fb32a2e572745)
cf73f598 Change for vulnerabilityChanges for vulnerability remediation
See merge request osdu/platform/system/indexer-queue!178
(cherry picked from commit b6ae94c5481cc5395fb86aae920fb32a2e572745)
cf73f598 Change for vulnerabilityShrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/180fix for dataformat-cbor vulnerabilty2022-03-24T07:55:17ZShrikant Gargfix for dataformat-cbor vulnerabiltyfix for whitesource vulnerabilityfix for whitesource vulnerabilityM11 - Release 0.14Anuj GuptaShrikant GargAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/182Merge branch 'fix-dataformat-vulnerability' into 'master'2022-03-24T07:58:45ZShrikant GargMerge branch 'fix-dataformat-vulnerability' into 'master'fix for dataformat-cbor vulnerabilty
See merge request osdu/platform/system/indexer-queue!180
(cherry picked from commit 63d20313911291f9c108d578e798007d48238b66)
9b6ad9f4 fix for dataformat-cbor vulnerabiltyfix for dataformat-cbor vulnerabilty
See merge request osdu/platform/system/indexer-queue!180
(cherry picked from commit 63d20313911291f9c108d578e798007d48238b66)
9b6ad9f4 fix for dataformat-cbor vulnerabiltyM11 - Release 0.14Anuj GuptaShrikant GargAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/184Fixed security issues related to jackson-databind2022-03-29T12:58:06ZRiabokon Stanislav(EPAM)[GCP]Fixed security issues related to jackson-databindUpdated to 2.11.4Updated to 2.11.4M11 - Release 0.14Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/183Upgrade First Party Library Dependencies for Release 0.142022-03-30T17:00:58ZDavid Diederichd.diederich@opengroup.orgUpgrade First Party Library Dependencies for Release 0.14This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 77b4144afb302aed5b64613b7a36e8c4ef324d7c
Maven: 0.14.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ----------------------------------------------------- | --------------- | ------------------------------------ |
| core-lib-azure | 0.14.0-rc5 | 0.13.0-rc6 |
| core-lib-gcp | 0.7.0, 0.13.0 | |
| os-core-lib-aws | 0.13.0 | |
| oqm | 0.13.0-SNAPSHOT | |
| os-core-common | 0.13.0 | 0.13.0-rc4 |
| os-core-lib-ibm | 0.13.0 | |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: eb15781b215dd2c4e6ca2ae92b621ab4a44b2084
Maven: 0.14.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ----------------------------------------------------- | --------------- | ------------------------------------ |
| core-lib-azure | 0.14.0 | 0.13.0-rc6 |
| core-lib-gcp | 0.7.0, 0.14.0 | |
| os-core-lib-aws | 0.14.0 | |
| obm | 0.14.0 | |
| oqm | 0.13.0-SNAPSHOT | |
| os-core-common | 0.14.0 | 0.13.0-rc4 |
| os-core-lib-ibm | 0.14.0 | |
| osm | 0.14.0 | |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |M11 - Release 0.14https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/187Cherry-pick "Upgrade First Party Library Dependencies for Release 0.14" into ...2022-03-30T23:14:55ZDavid Diederichd.diederich@opengroup.orgCherry-pick "Upgrade First Party Library Dependencies for Release 0.14" into release/0.14Original MR: !183Original MR: !183M11 - Release 0.14David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/188Cherry-pick "Upgrading dependency for OQM from a SNAPSHOT to a release versio...2022-03-30T23:15:02ZDavid Diederichd.diederich@opengroup.orgCherry-pick "Upgrading dependency for OQM from a SNAPSHOT to a release version" into release/0.14Original MR: !186Original MR: !186M11 - Release 0.14David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/189Cherry-pick "Upgrading core-lib-azure version for Testing Enqueue" into relea...2022-04-01T09:16:07ZDavid Diederichd.diederich@opengroup.orgCherry-pick "Upgrading core-lib-azure version for Testing Enqueue" into release/0.14Original MR: !185Original MR: !185M11 - Release 0.14David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/194Fixed pull subscription type activation condition2022-04-08T14:16:46ZDmitrii Novikov (EPAM)Fixed pull subscription type activation condition- Added `subscription-type=pull` property for anthos profile to activate OqmSubscriberManager- Added `subscription-type=pull` property for anthos profile to activate OqmSubscriberManagerM12 - Release 0.15Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/198update core-lib-gcp2022-05-01T06:53:56ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comupdate core-lib-gcpM12 - Release 0.15Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/207Update pom.xml2022-05-19T14:29:12ZShrikant GargUpdate pom.xmlUpdating core-lib versionUpdating core-lib versionM12 - Release 0.15Shrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/208update pom.xml2022-05-23T10:48:45ZAshwani Pandeyupdate pom.xmlupdate IBM core lib version to 0.15.0-rc2update IBM core lib version to 0.15.0-rc2M12 - Release 0.15Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/210fix auth header for istio2022-05-25T15:09:42ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comfix auth header for istioM12 - Release 0.15Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/214Upgrade First Party Library Dependencies for Release 0.152022-06-06T20:51:56ZDavid Diederichd.diederich@opengroup.orgUpgrade First Party Library Dependencies for Release 0.15This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Preparing packages...
Preparing packages...
Branch: master
SHA: f0dac823807b036c3d167e4a517fcda0f94fd061
Maven: 0.15.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ------------------------------------------------------- | ------------------------------ | ------------------------------------ |
| core-lib-azure | 0.15.0-rc6 | 0.14.0 |
| core-lib-gcp | 0.7.0, 0.15.0-rc3 | |
| os-core-lib-aws | 0.14.0 | |
| obm | 0.15.0-rc5 | |
| oqm | 0.15.0-rc2 | |
| os-core-common | 0.14.0, 0.15.0-rc6, 0.15.0-rc1 | 0.14.0 |
| os-core-lib-ibm | 0.15.0-rc2 | |
| osm | 0.15.0-rc5 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.2, 2.13.3, 2.11.4 | 2.13.2.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
### Dependency Information After the Upgrade
```
Preparing packages...
Preparing packages...
Branch: dependency-upgrade
SHA: 1c2aa68643474e5a6979d7a420f57e4fdcffa85f
Maven: 0.15.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ------------------------------------------------------- | ------------------------ | ------------------------------------ |
| core-lib-azure | 0.15.0 | 0.15.0 |
| core-lib-gcp | 0.7.0, 0.15.0 | |
| os-core-lib-aws | 0.15.0 | |
| obm | 0.15.0 | |
| oqm | 0.15.0 | |
| os-core-common | 0.15.0 | 0.15.0-rc4 |
| os-core-lib-ibm | 0.15.0 | |
| osm | 0.15.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.2, 2.13.3, 2.11.4 | 2.13.2.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |M12 - Release 0.15https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/220Upgrade Core IBM Library for Release 0.152022-06-16T05:14:09ZDavid Diederichd.diederich@opengroup.orgUpgrade Core IBM Library for Release 0.15This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 260df05673ac16cd53338c27b3b33a677edfca8e
Maven: 0.16.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ------------------------------------------------------- | ------------------------ | ------------------------------------ |
| core-lib-azure | 0.15.0 | 0.15.0 |
| core-lib-gcp | 0.7.0, 0.15.0 | |
| os-core-lib-aws | 0.15.0 | |
| obm | 0.15.0 | |
| oqm | 0.15.0 | |
| os-core-common | 0.15.0 | 0.15.0-rc4 |
| os-core-lib-ibm | 0.15.0 | |
| osm | 0.15.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.2, 2.13.3, 2.11.4 | 2.13.2.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
```
### Dependency Information After the Upgrade
```
Branch: core-ibm-upgrade
SHA: d8cff86e3167861cf1d00c3ab0e86a6722a1e62d
Maven: 0.16.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ------------------------------------------------------- | ------------------------ | ------------------------------------ |
| core-lib-azure | 0.15.0 | 0.15.0 |
| core-lib-gcp | 0.7.0, 0.15.0 | |
| os-core-lib-aws | 0.15.0 | |
| obm | 0.15.0 | |
| oqm | 0.15.0 | |
| os-core-common | 0.15.0 | 0.15.0-rc4 |
| os-core-lib-ibm | 0.15.2 | |
| osm | 0.15.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.2, 2.13.3, 2.11.4 | 2.13.2.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
```M12 - Release 0.15https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/221Cherry-pick 'Upgrade Core IBM Library for Release 0.15' into release/0.152022-06-16T14:50:05ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Upgrade Core IBM Library for Release 0.15' into release/0.15Original MR: !220Original MR: !220M12 - Release 0.15David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/218Upgrade First Party Library Dependencies for Release 0.152022-06-17T06:12:30ZDavid Diederichd.diederich@opengroup.orgUpgrade First Party Library Dependencies for Release 0.15This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Preparing packages...
Preparing packages...
Branch: master
SHA: 944131d56858986c5b8b393eeba03852a912a4b7
Maven: 0.15.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ------------------------------------------------------- | ------------------------ | ------------------------------------ |
| core-lib-azure | 0.15.0 | 0.15.0 |
| core-lib-gcp | 0.7.0, 0.15.0 | |
| os-core-lib-aws | 0.15.0 | |
| obm | 0.15.0 | |
| oqm | 0.15.0 | |
| os-core-common | 0.15.0 | 0.15.0-rc4 |
| os-core-lib-ibm | 0.15.0 | |
| osm | 0.15.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.2, 2.13.3, 2.11.4 | 2.13.2.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
```
### Dependency Information After the Upgrade
```
Preparing packages...
Preparing packages...
Branch: dependency-upgrade
SHA: ba7ca453c9c704605be9498df6753a68418957fb
Maven: 0.15.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/indexer-queue-azure-enqueue/ |
| ------------------------------------------------------- | ------------------------ | ------------------------------------ |
| core-lib-azure | 0.15.2 | 0.15.2 |
| core-lib-gcp | 0.7.0, 0.15.0 | |
| os-core-lib-aws | 0.15.0 | |
| obm | 0.15.0 | |
| oqm | 0.15.0 | |
| os-core-common | 0.15.0 | 0.15.0 |
| os-core-lib-ibm | 0.15.1 | |
| osm | 0.15.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.2.2, 2.13.3, 2.11.4 | 2.13.2.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
```
Critical: Found Vulnerable Jackson Databind dependency (<2.12.6.1 || >=2.13.0 <2.13.2.1)
```M12 - Release 0.15https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/222Cherry-pick 'Upgrade First Party Library Dependencies for Release 0.15' into ...2022-06-17T14:16:48ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Upgrade First Party Library Dependencies for Release 0.15' into release/0.15Original MR: !218Original MR: !218M12 - Release 0.15David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.org