Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2023-10-12T10:20:11Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/631merge code to gitlab2023-10-12T10:20:11ZYunhua Koglinmerge code to gitlabadd unit testsadd unit testsM21 - Release 0.24Derek HudsonDerek Hudsonhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/604Merge Delta changes from M16 master to M18 master2023-08-28T22:01:34ZVaibhavi KamaniMerge Delta changes from M16 master to M18 masterMerge Delta changes from M16 master to M18 masterMerge Delta changes from M16 master to M18 masterVaibhavi KamaniVaibhavi Kamanihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/638Merged the latest changes on M162023-10-17T20:05:21ZMahsa HanifiMerged the latest changes on M16Mahsa HanifiMahsa Hanifihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/403Merge ibm helm2022-10-10T06:02:13ZManish SinghMerge ibm helmMerge ibm helm to masterMerge ibm helm to masterM14 - Release 0.17Shrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/276Migrate Indexer Service to Anthos (GONRG-4090)2022-02-04T14:14:48ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comMigrate Indexer Service to Anthos (GONRG-4090)# Description:
Elasticsearch setting provisioning was moved to Partition service.
Refactoring.
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves...# Description:
Elasticsearch setting provisioning was moved to Partition service.
Refactoring.
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [x] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [x] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.M11 - Release 0.14Riabokon Stanislav(EPAM)[GCP]Andrei Dalhikh [EPAM/GC]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/471minor log fix2023-02-01T14:19:08ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comminor log fixfix logs for event processing.fix logs for event processing.M16 - Release 0.19Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/134Missed a SNAPSHOT dependency in the last MR..2023-08-18T15:08:19ZDavid Diederichd.diederich@opengroup.orgMissed a SNAPSHOT dependency in the last MR..Follow on to !129.Follow on to !129.M5 - Release 0.8David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/463Modify files for merging all helms indexer service2023-08-18T15:05:16ZVladyslav HundarchukModify files for merging all helms indexer serviceModify files for merging all helms for indexer service.Modify files for merging all helms for indexer service.M16 - Release 0.19Vladyslav HundarchukVladyslav Hundarchukhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/261move IndicesService & IMappingService interface back to Indexer service2022-09-16T08:32:39ZNeelesh Thakurmove IndicesService & IMappingService interface back to Indexer serviceRelated os-core-common MR: [136](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/136)
Move IndicesService & IMappingService interface back to Indexer core module. These are only two interfac...Related os-core-common MR: [136](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/136)
Move IndicesService & IMappingService interface back to Indexer core module. These are only two interfaces that were in os-core-common lib, all other Indexer interfaces are present in Indexer core module.
This change-set makes Indexer interfaces consistent and also remove Elastic dependencies from os-core-common. Elasticsearch is only used by Search and Indexer service but all os-core-common consumer services were inheriting Elasticsearch dependencies.M10 - Release 0.13https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/54Move Search Config out of os-core-common (GONRG-744)2023-08-18T15:17:27ZArtem Dobrynin (EPAM)Move Search Config out of os-core-common (GONRG-744)## Type of change
- [X] Feature
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does th...## Type of change
- [X] Feature
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does this introduce a breaking change?
- [YES]
The breaking change was introduced in [os-core-common](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/34)
The config from common lib does not support search- and indexer-related values. These values were moved to `org.opengroup.osdu.indexer.config.IndexerConfigurationProperties`
## What is the current behavior?
In this ticket the new separate config `org.opengroup.osdu.indexer.config.IndexerConfigurationProperties` was created. It is managed by `@ConfigurationProperties`.
All occurrences of the old config were replaced by new config
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful information
depends on:
https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/34
contains changes from: https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/52, https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/51M1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/436Moving the azure bootstrap config file to right package2022-12-14T09:46:48ZNaga Aneesh MylavarapuMoving the azure bootstrap config file to right packageIn the akprasad/unittest-util PR that got merged into main, the AzureBootstrapConfig file got moved to a wrong package causing indexer azure application to fail. This PR is raised to fix this issue and move the file to the right package.In the akprasad/unittest-util PR that got merged into main, the AzureBootstrapConfig file got moved to a wrong package causing indexer azure application to fail. This PR is raised to fix this issue and move the file to the right package.M16 - Release 0.19Naga Aneesh MylavarapuNaga Aneesh Mylavarapuhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/112Moving the while loop to iterate over storage records in azure code2023-08-18T15:08:41ZAman VermaMoving the while loop to iterate over storage records in azure code**Problem**
The reindexer service tries to call storage service for fetching records of a given kind. But the catch is that these calls are made in batches. In case the number of records for a given size are more than batch size, there ...**Problem**
The reindexer service tries to call storage service for fetching records of a given kind. But the catch is that these calls are made in batches. In case the number of records for a given size are more than batch size, there is no handling in place to read rest of the records. Hence each time we call reindex api for a certain kind **Only Tope 20 records are getting reindexed**
**FIX**
Putting the storage service related calls in a while loop.
```
Cursor = null
1. call the storage service with batch size = 20 and batch size
2. If no records are found for this kind then break
3. If there are records in current batch then put those records in indexer queue
4. If this was the last batch- terminate, else update the cursor to location of next batch and repeat from step 1
```
Tested in dev environment for one kind
cc: @manishk , @kiveerap
linked with https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/111?diff_id=8920M4 - Release 0.7Aman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/709[MS-34266] fix jetty vulnerability2024-02-12T15:17:57ZVidyaDharani Lokam[MS-34266] fix jetty vulnerability# Change details
* issue reference: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/145
https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/144
* updated `jetty` dependency to `6.1...# Change details
* issue reference: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/145
https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/144
* updated `jetty` dependency to `6.1.26` remediate vulnerability.
* excluded `jetty` from ibm csp as it is unused in it.
# Changes in:
* [x] Aws
* [x] Azure
* [x] GCP
* [x] IBM
* [x] Common codeM23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/354MSCOSDU-127: update manifests2023-08-18T15:05:27ZIgor Zimovets (EPAM)MSCOSDU-127: update manifestsChange motivation:
https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/docs/how-tos/minimize-downtime-during-deployments.mdChange motivation:
https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/docs/how-tos/minimize-downtime-during-deployments.mdM13 - Release 0.16https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/664[MSCOSDU-1851] upgrade netty and core common2023-12-12T17:31:46ZDeepa Kumari[MSCOSDU-1851] upgrade netty and core commonUpgrade:
1. core-lib-azure: upgraded to 0.25.0-rc2, upgrades org.json to 20231013
1. core-common: upgraded to 0.25.0-rc3
1. netty-version: from 4.1.98 to 4.1.101
2. spring-security-core: from 5.7.11 to 5.8.2Upgrade:
1. core-lib-azure: upgraded to 0.25.0-rc2, upgrades org.json to 20231013
1. core-common: upgraded to 0.25.0-rc3
1. netty-version: from 4.1.98 to 4.1.101
2. spring-security-core: from 5.7.11 to 5.8.2M22 - Release 0.25Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/700[MSCOSDU-1894] exclude unused maven-core dependency2024-01-25T17:01:10ZVidyaDharani Lokam[MSCOSDU-1894] exclude unused maven-core dependency# Change details
* reference issues: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/127 https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/126 https://community.opengroup.org/osd...# Change details
* reference issues: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/127 https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/126 https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/125 https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/124 https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/123
* excluded unused `maven-core` dependency to remediate vulnerability.
* upgraded `spring-security-core` to `5.8.9`
# Changes in:
* [x] GCP
* [x] Azure
* [x] AWS
* [ ] IBM
* [x] commoncodeM23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/688[MSCOSDU-1894] fix logback, json-smart and reactor-netty-http vulnerability2024-01-04T09:21:21ZVidyaDharani Lokam[MSCOSDU-1894] fix logback, json-smart and reactor-netty-http vulnerability* excluded `logback` dependencies and upgraded `log4j` version for Azure
* upgraded `json-smart` version to `2.5.0` to remediate vulnerability
* upgraded `reactor-netty-http` to `1.1.14` to remediate vulnerability* excluded `logback` dependencies and upgraded `log4j` version for Azure
* upgraded `json-smart` version to `2.5.0` to remediate vulnerability
* upgraded `reactor-netty-http` to `1.1.14` to remediate vulnerabilityM23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/691[MSCOSDU-1971] variablize and enable feature flags implemented2024-01-09T09:22:50ZDeepa Kumari[MSCOSDU-1971] variablize and enable feature flags implementedVariablize and enable feature flags for strategy, asIngestedCoordinates, keywordLower and bagOfWordsVariablize and enable feature flags for strategy, asIngestedCoordinates, keywordLower and bagOfWordsM22 - Release 0.25Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/719[#MSCOSDU-35931] upgrade : core-lib-azure Java 17 version2024-03-06T14:13:23ZThulasi Dass Subramanian[#MSCOSDU-35931] upgrade : core-lib-azure Java 17 version## Change details
* For Azure only: upgraded `core-lib-azure` version to `0.26.0-rc5` for Java 17 updates both in source & testing modules
## Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBM## Change details
* For Azure only: upgraded `core-lib-azure` version to `0.26.0-rc5` for Java 17 updates both in source & testing modules
## Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBMM23 - Release 0.26Thulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/29multi-tenancy updates for Indexer in azure2023-08-18T15:18:47ZDuvelis Caraomulti-tenancy updates for Indexer in azureDraft merge requests can't be merged.Draft merge requests can't be merged.M1 - Release 0.1