Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2023-07-25T07:11:22Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/510Vulnerability fix for IBM2023-07-25T07:11:22ZDevdatta SantraVulnerability fix for IBMVulnerability fix for IBM for tomcat and netty libs
| CVE-2022-41881 | https://nvd.nist.gov/vuln/detail/CVE-2022-41881 |
|----------------|-------------------------------------------------|
| CVE-2022-45143 | https://nvd.nist.gov/vuln/d...Vulnerability fix for IBM for tomcat and netty libs
| CVE-2022-41881 | https://nvd.nist.gov/vuln/detail/CVE-2022-41881 |
|----------------|-------------------------------------------------|
| CVE-2022-45143 | https://nvd.nist.gov/vuln/detail/CVE-2022-45143 |M17 - Release 0.20Devdatta SantraDevdatta Santrahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/509[GONRG-6649] Define global vars in indexer2023-08-18T15:05:04ZOleksandr Kosse (EPAM)[GONRG-6649] Define global vars in indexerM17 - Release 0.20https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/508Renaming dependency helm chart repo to osdu-aws and removing versioning const...2023-03-17T19:44:42ZMarc Burnie [AWS]Renaming dependency helm chart repo to osdu-aws and removing versioning constraintsM17 - Release 0.20Marc Burnie [AWS]Marc Burnie [AWS]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/506Upgraded azure lib to 20-rc5 for connected outer service endppoint versions2023-03-16T09:15:36ZDeepa KumariUpgraded azure lib to 20-rc5 for connected outer service endppoint versionsUpdated the core lib azure version to the one containing implementation of elastic endpoint versionsUpdated the core lib azure version to the one containing implementation of elastic endpoint versionsM17 - Release 0.20Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/505update docs with new Indexer acc config2023-03-13T13:19:41ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comupdate docs with new Indexer acc configM17 - Release 0.20Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/504Increase aws container memory2023-03-10T17:52:43ZManish JangidIncrease aws container memoryAws integration merge
See merge request osdu/platform/system/indexer-service!503
(cherry picked from commit 1f9a19bf63a334777194c651d5bef458555d5960)
5e55ab4c Allocating more memory to container than JVM heap memory
e5413d11 Merge rem...Aws integration merge
See merge request osdu/platform/system/indexer-service!503
(cherry picked from commit 1f9a19bf63a334777194c651d5bef458555d5960)
5e55ab4c Allocating more memory to container than JVM heap memory
e5413d11 Merge remote-tracking branch 'origin/master' into master-dev-mergeM16 - Release 0.19https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/503Aws integration merge2023-03-10T17:38:30ZManish JangidAws integration mergeUpdated container memoryUpdated container memoryM16 - Release 0.19https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/502Added logs for not found records2023-03-17T08:33:12ZDmitrii Novikov (EPAM)Added logs for not found records## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] I...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Added logs for `not found` records: due to missed metadata, missed during read from cloud storage and missded during conversion.
Some records were missed during indexing processM17 - Release 0.20Dmitrii Novikov (EPAM)Dmitrii Novikov (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/501Merge branch 'gc-upgrade-core-common' into 'master'2023-03-06T12:33:54ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comMerge branch 'gc-upgrade-core-common' into 'master'update core-common version
See merge request osdu/platform/system/indexer-service!500
(cherry picked from commit 0e10c4a10a629c35cf3ab6c06404bea6502d8c4c)
66af889a update core-common version
8d3770ba Updating NOTICEupdate core-common version
See merge request osdu/platform/system/indexer-service!500
(cherry picked from commit 0e10c4a10a629c35cf3ab6c06404bea6502d8c4c)
66af889a update core-common version
8d3770ba Updating NOTICEM16 - Release 0.19Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/500update core-common version2023-03-07T08:57:12ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comupdate core-common versionUpdate core-common version to fix ElasticIndexNameResolver.isIndexAliasSupported method not found exception.Update core-common version to fix ElasticIndexNameResolver.isIndexAliasSupported method not found exception.M16 - Release 0.19Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/498Enable geo-shape decimation for all the kinds that have spatial location2023-03-08T14:42:39ZZhibin MaiEnable geo-shape decimation for all the kinds that have spatial locationThis is follow up of MR [Geoshape decimation](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/387) and MR [Enable the feature flag for geo-shape decimation by default](https://community.opengroup.org...This is follow up of MR [Geoshape decimation](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/387) and MR [Enable the feature flag for geo-shape decimation by default](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/486) to enable geo-shape decimation for all the kinds that have spatial data.
The previous implementation only enables the geo-shape decimation for the kinds that have "data.VirtualProperties.DefaultLocation" defined in their schema. One of our clients tested the shape decimation for Seismic2D survey data and they found that the map performance was increased 20-30 times after this feature is enabled and data of the Seismic2D survey were re-indexed.M17 - Release 0.20Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/497Gc rabbitmq recovery2023-03-06T16:50:41ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comGc rabbitmq recoveryM17 - Release 0.20Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/496Provide an endpoint to create aliases for all existing indices that do not ha...2023-05-11T12:33:50ZZhibin MaiProvide an endpoint to create aliases for all existing indices that do not have aliasesThis is follow-up of fix for issue [Multi kind search does not work with more than approximately 85 kinds](https://community.opengroup.org/osdu/platform/system/search-service/-/issues/108).
In this MR, we add an endpoint that allows clo...This is follow-up of fix for issue [Multi kind search does not work with more than approximately 85 kinds](https://community.opengroup.org/osdu/platform/system/search-service/-/issues/108).
In this MR, we add an endpoint that allows cloud operators to create aliases for all existing indices. Doing that, queries from a given data partition won't trigger the alias creation operation in the search service after the endpoint is called once for the given data partition.
Please note that previous change in search service still works even the cloud operators don't call this new endpoint if they don't have time now. However, it is still recommended to explicitly call it once for all existing data partitions.M18 - Release 0.21Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/495Fix redis-indexer cpu request2023-08-18T15:05:06ZOleksandr Kosse (EPAM)Fix redis-indexer cpu requestM17 - Release 0.20https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/494Update default requsts/limits for indexer service2023-08-18T15:05:08ZOleksandr Kosse (EPAM)Update default requsts/limits for indexer serviceM17 - Release 0.20https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/493Cherry-pick 'Security fixes AWS' into release/0.192023-02-17T20:36:42ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Security fixes AWS' into release/0.19**Original MR**: !489
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !489
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/new?ref=cherry-pick-for-489)M16 - Release 0.19David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/492Cherry-pick 'Remove SNAPSHOT dependencies' into release/0.192023-02-18T07:29:34ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Remove SNAPSHOT dependencies' into release/0.19**Original MR**: !491
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !491
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/new?ref=cherry-pick-for-491)M16 - Release 0.19David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/491Remove SNAPSHOT dependencies2023-02-18T07:29:34ZDavid Diederichd.diederich@opengroup.orgRemove SNAPSHOT dependenciesThis automated MR removes usage of `SNAPSHOT` versions in the first party library dependencies.
Since `SNAPSHOT` dependencies change frequently -- by their nature -- usage of them across projects is dangerous and should be avoided.
### ...This automated MR removes usage of `SNAPSHOT` versions in the first party library dependencies.
Since `SNAPSHOT` dependencies change frequently -- by their nature -- usage of them across projects is dangerous and should be avoided.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 58e2828de09ee79e5a399bb6dbfcf8d11eb030e2
Maven: 0.20.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | --------------------------- | ---------------------- |
| core-lib-azure | 0.18.0-rc3 | 0.12.0-rc10 |
| core-lib-gcp | 0.19.0-rc7 | |
| os-core-lib-aws | 0.18.0 | 0.14.0-rc2 |
| obm | 0.19.0-rc4 | |
| oqm | 0.19.0-rc4 | |
| os-core-common | 0.19.0-SNAPSHOT, 0.19.0-rc3 | 0.14.0-rc8 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.13.0 |
| osm | 0.18.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.4.2, 2.13.2.2, 2.13.4 | 2.13.2.2, 2.13.2 |
| (3rd Party) net.minidev.json-smart | 2.4.7 | 2.4.6 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.11.1, 2.11.2, 2.13.0 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.11.2, 2.13.0 |
| (3rd Party) org.springframework.spring-webflux | 5.3.22 | |
| (3rd Party) org.springframework.spring-webmvc | 5.3.24, 5.3.22 | 5.3.23 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade-4
SHA: 4f1e194e9fad9e6608e1890630925267ea1ef856
Maven: 0.20.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | ---------------------- |
| core-lib-azure | 0.18.0-rc3 | 0.12.0-rc10 |
| core-lib-gcp | 0.19.0-rc7 | |
| os-core-lib-aws | 0.18.0 | 0.14.0-rc2 |
| obm | 0.19.0-rc4 | |
| oqm | 0.19.0-rc4 | |
| os-core-common | 0.19.0 | 0.14.0-rc8 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.13.0 |
| osm | 0.18.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.13.4.2, 2.13.2.2, 2.13.4 | 2.13.2.2, 2.13.2 |
| (3rd Party) net.minidev.json-smart | 2.4.7 | 2.4.6 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.11.1, 2.11.2, 2.13.0 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.11.2, 2.13.0 |
| (3rd Party) org.springframework.spring-webflux | 5.3.22 | |
| (3rd Party) org.springframework.spring-webmvc | 5.3.24, 5.3.22 | 5.3.23 |M16 - Release 0.19https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/490Delete vars for iq [GONRG-6462]2023-08-18T15:05:10ZMikhail Piatliou (EPAM)Delete vars for iq [GONRG-6462]M17 - Release 0.20Mikhail Piatliou (EPAM)Mikhail Piatliou (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/489Security fixes AWS2023-02-17T20:35:45ZYash DholakiaSecurity fixes AWScommit bbcc6995
Author: Abhay <bios@amazon.com>
Date: Fri Dec 02 2022 10:44:20 GMT-0800 (Pacific Standard Time)
upgrade dependenciescommit bbcc6995
Author: Abhay <bios@amazon.com>
Date: Fri Dec 02 2022 10:44:20 GMT-0800 (Pacific Standard Time)
upgrade dependenciesM16 - Release 0.19Yash DholakiaYash Dholakia