Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2021-03-25T17:21:26Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/69Fix typo in IndexerService.md2021-03-25T17:21:26ZSviatoslav NekhaienkoFix typo in IndexerService.mdFix typoFix typohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/45fix unexpected conversion from int to double2021-07-01T01:39:28ZSviatoslav Nekhaienkofix unexpected conversion from int to doubleGson parser parses any integer value as Double,
Jackson is smarter, it uses Integer of BigIntegerGson parser parses any integer value as Double,
Jackson is smarter, it uses Integer of BigIntegerSviatoslav NekhaienkoSviatoslav Nekhaienkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/518Full Upgrade of First Party Library Dependencies for Release 0.202023-05-22T15:18:25ZDavid Diederichd.diederich@opengroup.orgFull Upgrade of First Party Library Dependencies for Release 0.20This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will ...This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will often fail, since the upgrades were previously rejected for failing pipelines and have not been directly addressed yet.
This upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
This MR may co-exist with a separate, smaller upgrade MR.
If both pass, this one should be used instead.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 96df4fc279ee82ede9c052d2411e77ef7f9dd177
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | ---------------------- |
| core-lib-azure | 0.20.0-rc5 | 0.12.0-rc10 |
| core-lib-gcp | 0.20.0-rc2 | |
| os-core-lib-aws | 0.21.0-rc1 | 0.19.0-rc3 |
| obm | 0.19.0 | |
| oqm | 0.19.0 | |
| os-core-common | 0.19.0, 0.19.0-rc6 | 0.14.0-rc8 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.13.0 |
| osm | 0.20.0-rc2 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.14.1, 2.13.2.2, 2.13.4.2 | 2.13.2.2, 2.13.2 |
| (3rd Party) net.minidev.json-smart | 2.4.7 | 2.4.6 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.11.1, 2.11.2, 2.13.0 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.11.2, 2.13.0 |
| (3rd Party) org.yaml.snakeyaml | 2.0 | 1.26 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: 5d54b9fa233d653d1d2a69fc420d1472a5eee660
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------------------------------- | -------------------------- | ---------------------- |
| core-lib-azure | 0.20.0 | 0.20.0 |
| core-lib-gc | 0.20.0 | |
| os-core-lib-aws | 0.21.0-rc2 | 0.21.0-rc2 |
| obm | 0.20.0 | |
| oqm | 0.20.0 | |
| os-core-common | 0.20.1 | 0.20.1 |
| os-core-lib-ibm | 0.20.0 | 0.20.0 |
| osm | 0.20.0 | |
| (3rd Party) com.fasterxml.jackson.core.jackson-databind | 2.14.1, 2.13.2.2, 2.13.4.2 | 2.13.2.2, 2.13.2 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.11.1, 2.11.2, 2.13.0 |
| (3rd Party) org.apache.logging.log4j.log4j-to-slf4j | 2.17.1 | 2.11.2, 2.13.0 |
| (3rd Party) org.yaml.snakeyaml | 2.0 | 1.26, 1.27, 2.0 |M18 - Release 0.21Okoun-Ola Fabien HouetoBrindaban DasYauhen Shaliou [EPAM/GCP]Srinivasan NarayananOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/499GC update core-common version2023-03-06T08:34:47ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comGC update core-common versionUpdate core-common version to fix ElasticIndexNameResolver.isIndexAliasSupported method not found exception.Update core-common version to fix ElasticIndexNameResolver.isIndexAliasSupported method not found exception.M16 - Release 0.19Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/202Gonrg 3158 switch schema endpoints for all related services2021-09-21T21:44:08ZOleksandr Kosse (EPAM)Gonrg 3158 switch schema endpoints for all related servicesOleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/295(GONRG-4427) helm on prem2022-04-08T13:36:03ZVladislav Shishko (EPAM)(GONRG-4427) helm on premVladislav Shishko (EPAM)Vladislav Shishko (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/427GONRG-6045: Remove NEG annotation in service2022-12-06T11:30:53ZVolodymyr Pienskoi [EPAM / GCP]GONRG-6045: Remove NEG annotation in serviceServices created in GKE clusters 1.17.6-gke.7 and up with VPC-native traffic routing enabled are annotated automatically with `cloud.google.com/neg: '{"ingress": true}'`. This means that this annotation is not required explicitly and can...Services created in GKE clusters 1.17.6-gke.7 and up with VPC-native traffic routing enabled are annotated automatically with `cloud.google.com/neg: '{"ingress": true}'`. This means that this annotation is not required explicitly and can be removed.
More details in [GKE Load Balancing documentation](https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#container-native_load_balancing).Volodymyr Pienskoi [EPAM / GCP]Volodymyr Pienskoi [EPAM / GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/111Handling the storage api calls correctly2021-03-05T11:54:18ZAman VermaHandling the storage api calls correctly**Problem**
The reindexer service tries to call storage service for fetching records of a given kind. But the catch is that these calls are made in batches. In case the number of records for a given size are more than batch size, there ...**Problem**
The reindexer service tries to call storage service for fetching records of a given kind. But the catch is that these calls are made in batches. In case the number of records for a given size are more than batch size, there is no handling in place to read rest of the records. Hence each time we call reindex api for a certain kind **Only Tope 20 records are getting reindexed**
**FIX**
Putting the storage service related calls in a while loop.
```
Cursor = null
1. call the storage service with batch size = 20 and batch size
2. If no records are found for this kind then break
3. If there are records in current batch then put those records in indexer queue
4. If this was the last batch- terminate, else update the cursor to location of next batch and repeat from step 1
```
Tested in dev environment for one kind
cc: @manishk , @kiveerapAman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/682hotfix2023-12-22T08:07:01ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhotfixStanisław BienieckiStanisław Bienieckihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/27IBM impl index enhancement2020-10-19T11:50:21ZAnuj GuptaIBM impl index enhancementIBM Indexer enhancements for indexerIBM Indexer enhancements for indexerAnuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/40ibm sync with develop2020-10-16T09:58:13ZGokul Nagareibm sync with developAnuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/43Improve code coverage (indexer-azure)2021-11-19T12:05:31ZAalekh JainImprove code coverage (indexer-azure)## All Submissions:
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [YES] All new ...## All Submissions:
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [NO] I ran lint checks locally prior to submission.
## What is the current behavior?
No Unit Tests and zero code coverage for indexer-azure.
## What is the new behavior?
Added Unit Tests to improve code coverage for indexer-azure
## Does this introduce a breaking change?
- [NO]
cc: @kibattul @polavishnu @amaverma
# Note
To exclude the Config classes from jacoco code coverage reports, add the following configuration in pom.xml for Jacoco plugin.
```xml
<configuration>
<excludes>
<exclude>org/opengroup/osdu/indexer/azure/*/*Config*</exclude>
</excludes>
</configuration>
```https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/439jar type vulnerability fix for indexer-service2023-02-16T09:54:09ZPintu Guptajar type vulnerability fix for indexer-serviceFollowing CVE has been fixed into this MR :
| cve | link |
|----------------|-------------------------------------------------|
| CVE-2022-42003 | https://nvd.nist.gov/vuln/detail/CV...Following CVE has been fixed into this MR :
| cve | link |
|----------------|-------------------------------------------------|
| CVE-2022-42003 | https://nvd.nist.gov/vuln/detail/CVE-2022-42003 |
| CVE-2022-42004 | https://nvd.nist.gov/vuln/detail/CVE-2022-42004 |
| CVE-2022-25857 | https://nvd.nist.gov/vuln/detail/CVE-2022-25857 |
| CVE-2022-42252 | https://nvd.nist.gov/vuln/detail/CVE-2022-42252 |Pintu GuptaPintu Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/183Locking down maven central2021-08-05T09:46:02ZDzmitry_Paulouski (slb)Locking down maven centralcommit 54e13552
Author: Spencer Sutton <suttonsp@amazon.com>
Date: Thu May 27 2021 10:10:45 GMT-0500 (Central Daylight Time)
"Excluding springs default security, locking down mvn central"commit 54e13552
Author: Spencer Sutton <suttonsp@amazon.com>
Date: Thu May 27 2021 10:10:45 GMT-0500 (Central Daylight Time)
"Excluding springs default security, locking down mvn central"https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/247log4j vuln fixed2021-12-16T07:45:16ZGokul Nagarelog4j vuln fixedfixes issue -
| module pom changes | Ref Issue |
|--------------------|----------------------------------------------------------------------------------...fixes issue -
| module pom changes | Ref Issue |
|--------------------|-----------------------------------------------------------------------------------------------|
| IBM | https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm/-/issues/2 |
| Core | https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/54 |Shrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/417M14 upgrade2022-11-16T13:01:12ZAnkur RawatM14 upgradeM12 changes into M14M12 changes into M14Ankur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/580M18 azure preship debug2023-08-25T22:29:40ZZhibin MaiM18 azure preship debugadd debug codes to debug augmenteradd debug codes to debug augmenterZhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/600Merge Delta changes from M16 master to M18 master2023-08-28T22:02:46ZVaibhavi KamaniMerge Delta changes from M16 master to M18 masterMerge Delta changes from M16 master to M18 masterMerge Delta changes from M16 master to M18 masterVaibhavi KamaniVaibhavi Kamanihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/599Merge Delta changes from M16 to M18 master2023-08-23T19:54:17ZVaibhavi KamaniMerge Delta changes from M16 to M18 master
Merge Delta changes from M16 to M18
Merge Delta changes from M16 to M18Vaibhavi KamaniVaibhavi Kamanihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/598Merged from M162023-08-16T21:44:13ZVaibhavi KamaniMerged from M16Merged from M16Merged from M16Vaibhavi KamaniVaibhavi Kamani