Search merge requestshttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests2024-03-20T12:42:07Zhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/633[GONRG-9422]: Remove springProfilesActive explicit setting2024-03-20T12:42:07ZMykola Ronik [EPAM / GCP][GONRG-9422]: Remove springProfilesActive explicit setting## Type of change
- [ ] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
https://jiraeu.epam.com/browse/GONRG-9422
## Does this introduce a change in the core logic?
- [NO]
## Does...## Type of change
- [ ] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
https://jiraeu.epam.com/browse/GONRG-9422
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]M23 - Release 0.26Mykola Ronik [EPAM / GCP]Mykola Ronik [EPAM / GCP]https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/632search service community implementation2024-03-14T09:56:16ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comsearch service community implementation# Description:
Search Service Community Implementation.
Link: https://gitlab.opengroup.org/osdu/pmc/community-implementation/-/issues/9
- Added a new module Seach Core Plus.
- Removed GC-specific dependencies.
- Added APD driver depende...# Description:
Search Service Community Implementation.
Link: https://gitlab.opengroup.org/osdu/pmc/community-implementation/-/issues/9
- Added a new module Seach Core Plus.
- Removed GC-specific dependencies.
- Added APD driver dependency.
- APD-OpenID implementation is optional, and could be used when Partition service requires authorization.
# How to test:
Integration tests.
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [x] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] Community Implementation
# Dev Checklist:
- [x] Added Unit Tests, wherever applicable.
- [x] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.M23 - Release 0.26Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/631The search service info endpoint is not error-tolerant (GONRG-9391)2024-03-12T08:10:54ZRiabokon Stanislav(EPAM)[GCP]The search service info endpoint is not error-tolerant (GONRG-9391)## Type of change
- [x] Bug Fix
- [ ] Feature
https://community.opengroup.org/osdu/platform/system/search-service/-/issues/159
## Does this introduce a change in the core logic?
- [YES/NO]
## Does this introduce a change in the cloud...## Type of change
- [x] Bug Fix
- [ ] Feature
https://community.opengroup.org/osdu/platform/system/search-service/-/issues/159
## Does this introduce a change in the core logic?
- [YES/NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [x] Azure
- [x] Google Cloud
- [x] IBM
## Does this introduce a breaking change?
- [YES]M23 - Release 0.26Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/628Refactor tests and add unit test2024-03-10T00:11:05ZSolomon AyalewRefactor tests and add unit testIncrease test coverage and remove duplicate codeIncrease test coverage and remove duplicate codeM23 - Release 0.26https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/627[GONRG-9211] Added labels to gc helm2024-03-19T17:13:15ZDanylo Vanin (EPAM)[GONRG-9211] Added labels to gc helm## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] Google Clou...## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]M23 - Release 0.26Danylo Vanin (EPAM)Danylo Vanin (EPAM)https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/626Fix gc test2024-03-05T15:07:03ZSiarhei Poliak [EPAM / GCP]Fix gc test## Type of change
- [ ] Bug Fix
- [x] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provi...## Type of change
- [ ] Bug Fix
- [x] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]M23 - Release 0.26Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/625[#MSCOSDU-35937] upgrade : core-lib-azure Java 17 version2024-03-07T07:45:04ZThulasi Dass Subramanian[#MSCOSDU-35937] upgrade : core-lib-azure Java 17 version## Change details
* For Azure only: upgraded `core-lib-azure` version to `0.26.0-rc5` for Java 17 updates both in source & testing modules
## Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBM## Change details
* For Azure only: upgraded `core-lib-azure` version to `0.26.0-rc5` for Java 17 updates both in source & testing modules
## Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBMM23 - Release 0.26Thulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/624Add code lost in the rebase process2024-03-08T11:17:29ZStanisław BienieckiAdd code lost in the rebase process## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [YES/NO]
## Does this introduce a change in the cloud p...## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [YES/NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [YES/NO]
## What is the current behavior?
## What is the new/expected behavior?
## Have you added/updated Unit Tests and Integration Tests?
## Any other useful informationM23 - Release 0.26https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/623Update os-core-lib-aws2024-03-19T17:13:17ZRucha DeshpandeUpdate os-core-lib-aws## Type of change
- [X ] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud prov...## Type of change
- [X ] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X ] AWS
- [ ] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
AWS Only change. Updating os-core-lib-aws
## What is the new/expected behavior?
## Have you added/updated Unit Tests and Integration Tests?
## Any other useful informationM23 - Release 0.26https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/622[34894] fix integration tests in Azure2024-02-12T06:11:13ZVidyaDharani Lokam[34894] fix integration tests in Azure## Type of change
* [x] Bug Fix
* [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
* \[NO\]
## Does this introduce a change in the cloud prov...## Type of change
* [x] Bug Fix
* [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
* \[NO\]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
* [x] Azure
* [ ] AWS
* [ ] Google Cloud
* [ ] IBM
## Does this introduce a breaking change?
* \[NO\]
## What is the current behavior?
Integrations tests are currently broken on Azure following the enabling of the BagOfWords feature, which increase the number of results returned by a "text" query as the ES Analyzer changed.
## What is the new/expected behavior?
Update the tests cases to force the query on a specific text field, thus still using exact matched and returning the correct number of expected results
## Have you added/updated Unit Tests and Integration Tests?
Yes Related change on Indexer which introduce the failure: https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/649M23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/619AWS - Fix integration tests2024-02-01T15:05:37ZGuillaume CailletAWS - Fix integration testsCherry pick of https://community.opengroup.org/osdu/platform/system/search-service/-/commit/e1d3f7f175917d97cfbfcf60122b4d35dd427693
## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [N...Cherry pick of https://community.opengroup.org/osdu/platform/system/search-service/-/commit/e1d3f7f175917d97cfbfcf60122b4d35dd427693
## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [ ] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Current integrations tests for AWS are broken following the bag of word feature.
This cherry pick the fix in the tests from masterM22 - Release 0.25Guillaume CailletGuillaume Caillethttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/618Fix IBM compile step failing because of non existing parent version2024-02-01T15:06:02ZGuillaume CailletFix IBM compile step failing because of non existing parent version## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] Google Cloud...## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] Google Cloud
- [X] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Current pipeline on release/0.25 branch is broken, the compile step is failing because of missing parents: https://community.opengroup.org/osdu/platform/system/search-service/-/pipelines/240114M22 - Release 0.25Guillaume CailletGuillaume Caillethttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/617Set redis timeout to 60s2024-03-19T17:13:19ZNeha KhandelwalSet redis timeout to 60s## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- \[YES/**NO**\]
## Does this introduce a change in the cl...## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- \[YES/**NO**\]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- \[YES/**NO**\]
Currently, the Redis cache timeout is incorrectly set for 1hr. Elasticsearch backend cursors are only valid for 60s. They are hashed and cached by search service for security reasons. Since the cache timeout does not make that of the backend cursors, this causes an additional call to Elasticsearch that eventually fails. M23 - Release 0.26https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/616AWS - Merge from dev2024-01-24T16:43:05ZGuillaume CailletAWS - Merge from devAWS Change only.
Empty diff, but this should fix our automated sync scripts from detecting changes between Gitlab and CodeCommit.AWS Change only.
Empty diff, but this should fix our automated sync scripts from detecting changes between Gitlab and CodeCommit.M23 - Release 0.26Guillaume CailletGuillaume Caillethttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/615Fix AWS integration tests after BagofWords feature2024-01-19T19:19:02ZGuillaume CailletFix AWS integration tests after BagofWords feature## Type of change
- [X] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provi...## Type of change
- [X] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [ ] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Integrations tests are currently broken on AWS following the enabling of the BagOfWords feature, which increase the number of results returned by a "text" query as the ES Analyzer changed.
## What is the new/expected behavior?
Update the tests cases to force the query on a specific text field, thus still using exact matched and returning the correct number of expected results
## Have you added/updated Unit Tests and Integration Tests?
Yes
Related change on Indexer which introduce the failure: https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/649M23 - Release 0.26Guillaume CailletGuillaume Caillethttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/614Move QueryBuilders to Core2024-01-29T12:41:03ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comMove QueryBuilders to Core# Description:
ADR: https://community.opengroup.org/osdu/platform/system/search-service/-/issues/154
The Search service contains duplicated code for constructing Elasticsearch queries within CSP modules, in classes such as QueryBase.jav...# Description:
ADR: https://community.opengroup.org/osdu/platform/system/search-service/-/issues/154
The Search service contains duplicated code for constructing Elasticsearch queries within CSP modules, in classes such as QueryBase.java and QueryServiceImpl.java. These redundancies add complexity to code maintenance without offering visible benefits. Query builders do not have CSP-specific code.
Differences have emerged in these classes over time, most notable that was affected during refactoring:
- Azure [DependencyLogger](https://community.opengroup.org/osdu/platform/system/search-service/-/blob/master/provider/search-azure/src/main/java/org/opengroup/osdu/search/provider/azure/utils/DependencyLogger.java) was refactored, and to the Search core was added an Interface to abstract this logger, with the default implementation.
- The AWS QueryBuilder involves mandatory ID prefix injection into the query, which appears contradictory to the Search API that limits results. The injection of the ID prefix was not moved to the Core Query Builder: <br/>
`queryBuilder.must(prefixQuery("id", String.format("%s:", this.dpsHeaders.getPartitionId())));` <br/>
In the Elastisearch request, it looked like this:
~~~
"bool": {
"must": [
{
"prefix": {
"id": {
"value": "opendes:",
"boost": 1.0
}
}
}
],
~~~
# How to test:
Integration tests.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GC
- [x] Azure
- [x] AWS
- [x] IBM
# Dev Checklist:
- [x] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:
It is possible to roll back CSP migration if the agreement isn't met.
The QueryBuilder in Search Core would then become optional, and CSP could continue using their own QueryBuilder.M23 - Release 0.26Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/613Upgrade core-lib-ibm version2024-01-05T08:45:45ZAshwani PandeyUpgrade core-lib-ibm versionhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/610https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/610M22 - Release 0.25Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/612reduce the group cache timeout to 30s2024-01-09T02:02:02ZNeelesh Thakurreduce the group cache timeout to 30s## Type of change
- [ ] Bug Fix
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
Reduce the...## Type of change
- [ ] Bug Fix
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
Reduce the group cache TTL to 30s. Currently it's set to 1hr, which is very long time. Record with newly created group/ACL won't be searchable for 1hr. This change also makes this TTL consistent with Storage service cache configuration.M23 - Release 0.26Neelesh ThakurChad LeongNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/611[MSCOSDU-1894] fix json-smart vulnerability2024-01-03T07:24:23ZVidyaDharani Lokam[MSCOSDU-1894] fix json-smart vulnerability* upgraded `json-smart` to `2.5.0` to remediate vulnerability.
* upgraded `reactor-netty` to `1.1.14`.* upgraded `json-smart` to `2.5.0` to remediate vulnerability.
* upgraded `reactor-netty` to `1.1.14`.M23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/610ibm-lib version upgrade2024-01-05T08:37:10ZAshwani Pandeyibm-lib version upgradecore-lib-ibm version upgrade to 0.25.1core-lib-ibm version upgrade to 0.25.1M22 - Release 0.25Ashwani PandeyAshwani Pandey