Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2023-08-18T15:17:27Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/54Move Search Config out of os-core-common (GONRG-744)2023-08-18T15:17:27ZArtem Dobrynin (EPAM)Move Search Config out of os-core-common (GONRG-744)## Type of change
- [X] Feature
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does th...## Type of change
- [X] Feature
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does this introduce a breaking change?
- [YES]
The breaking change was introduced in [os-core-common](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/34)
The config from common lib does not support search- and indexer-related values. These values were moved to `org.opengroup.osdu.indexer.config.IndexerConfigurationProperties`
## What is the current behavior?
In this ticket the new separate config `org.opengroup.osdu.indexer.config.IndexerConfigurationProperties` was created. It is managed by `@ConfigurationProperties`.
All occurrences of the old config were replaced by new config
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful information
depends on:
https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/34
contains changes from: https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/52, https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/51M1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/53fix whistesource issues for the indexer-core and indexer-azure;2023-08-18T15:17:29ZYauheni Lesnikaufix whistesource issues for the indexer-core and indexer-azure;Fix whistesource issues for the indexer-core and indexer-azure;
Spring-boot version increased to the latest one of the 2.1.x series.
com.fasterxml.woodstox:woodstox-core - increased to the latest oneFix whistesource issues for the indexer-core and indexer-azure;
Spring-boot version increased to the latest one of the 2.1.x series.
com.fasterxml.woodstox:woodstox-core - increased to the latest oneM1 - Release 0.1ethiraj krishnamanaiduNeelesh ThakurDuvelis CaraoYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/52Gcp support http target cloud tasks (GONRG-1041)2023-08-18T15:17:31ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comGcp support http target cloud tasks (GONRG-1041)## Type of change
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does thi...## Type of change
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
When cloud task hit indexer service, to authenticate it, indexer verify that it have only specified in core-lib-gcp headers
## What is the new/expected behavior?
When cloud task hit indexer service, to authenticate it, indexer verify that it have specified in core-lib-gcp headers,
and if this task not AppEngine task , token provided with task also must be verified.
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful information
depends on:
https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/50
https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp/-/merge_requests/6
have changes from:
https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/51M1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/51GCP add integration test for index cleanup (GONRG-1024)2023-08-18T15:17:33ZIgor Filippov (EPAM)GCP add integration test for index cleanup (GONRG-1024)## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] I...## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
A controller does not have an integration test.
## What is the new/expected behavior?
An integration test has been added for CleanupIndiciesApi controller.
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful informationM1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/50Removed unhardcode repositories and distribution management from all poms (GO...2023-08-18T15:17:35ZRiabokon Stanislav(EPAM)[GCP]Removed unhardcode repositories and distribution management from all poms (GONRG-1056)## Type of change
- [ ] Bug Fix
- [X] Feature
ADR: https://community.opengroup.org/osdu/platform/system/home/-/issues/55
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider...## Type of change
- [ ] Bug Fix
- [X] Feature
ADR: https://community.opengroup.org/osdu/platform/system/home/-/issues/55
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does this introduce a breaking change?
- [YES]
## What is the new/expected behavior?
This MR brings changes to POMs (root, azure, gcp) to improve (unhardcode) the way how repositories are being pointed. It now uses set of properties defined in the `.mvn/community-maven.settings.xml`
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful informationM1 - Release 0.1Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/49Updated Helm chart2023-08-18T15:18:16ZDaniel SchollUpdated Helm chartChanging Default Sizing of Standard Azure Deployment. https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/42Changing Default Sizing of Standard Azure Deployment. https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/42M1 - Release 0.1Daniel SchollDaniel Schollhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/47Azure SDKs Upgrade and Dependency Management2023-08-18T15:18:29ZAbhishek PatilAzure SDKs Upgrade and Dependency Management## All Submissions:
-------------------------------------
* [YES] Have you followed our code review [guidelines](https://github.com/microsoft/code-with-engineering-playbook/blob/master/pull-requests/code-reviews/readme.md)?
* [YES] Have ...## All Submissions:
-------------------------------------
* [YES] Have you followed our code review [guidelines](https://github.com/microsoft/code-with-engineering-playbook/blob/master/pull-requests/code-reviews/readme.md)?
* [YES] Have you added an explanation of what your changes do and why you'd like us to include them?
* [YES] I have updated the documentation accordingly.
* [NA] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [NO] I ran lint checks locally prior to submission.
## What is the current behavior?
-------------------------------------
- Old version of Azure SDKs are used in service.
- Versions of dependencies which are common across all OSDU services are not consistent/same in all OSDU services.<!-- Please describe the current behavior that you are modifying, or link to a relevant issue. -->
Issue: https://dev.azure.com/msazure/One/_workitems/edit/8708950
## What is the new behavior?
-------------------------------------
- All Azure SDKs are upgraded to latest stable version.
- Common dependencies are inherited from core-lib-azure as managed dependencies. This will make sure that version info of those dependencies are inherited from core-lib-azure. All OSDU services using same version of core-lib-azure will automatically use same versions for all those dependencies.<!-- Please describe the behavior or changes that are being added by this PR. -->
## Does this introduce a breaking change?
-------------------------------------
- [NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Any relevant logs, error output, etc?
-------------------------------------
(If it’s long, please paste to https://ghostbin.com/ and insert the link here.)
## Other information
-------------------------------------
<!-- Any other information that is important to this PR such as screenshots of how the component looks before and after the change. -->M1 - Release 0.1Abhishek PatilAbhishek Patilhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/46GCP fix swagger (GONRG-1022)2023-08-18T15:18:31ZIgor Filippov (EPAM)GCP fix swagger (GONRG-1022)## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce new features/functionality in core logic?
- [No]
## Does this introduce a breaking change?
- [No]
## Doe...## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce new features/functionality in core logic?
- [No]
## Does this introduce a breaking change?
- [No]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IBM
## What is the current behavior?
Swagger doesn't work in GCP.
## What is the new/expected behavior?
Swagger is working in GCP.
## Have you added/updated Unit Tests and Integration Tests?
- [No]
## Any other useful information
Changes:
- Change some code in SwaggerDocumentationConfig classM1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/42Ibm api key test2023-08-18T15:18:32ZGokul NagareIbm api key testM1 - Release 0.1Anuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/41GCP implement index cleanup (GONRG-856)2023-08-18T15:18:34ZIgor Filippov (EPAM)GCP implement index cleanup (GONRG-856)## Type of change
- [ ] Bug Fix
- [X] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/5#note_16693
## Does this intro...## Type of change
- [ ] Bug Fix
- [X] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/5#note_16693
## Does this introduce a change in the core logic?
- [No]
## Does this introduce new features/functionality in core logic?
- [YES]
## Does this introduce a breaking change?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [x] Azure
- [x] GCP
- [x] IBM
## What is the current behavior?
GCP provider don't clean schema indexes when schema deletes.
## What is the new/expected behavior?
GCP provider clean schema indexes when schema deletes.
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful information
Changes:
- Add new controller CleanupIndiciesApi
- Add new test class CleanupIndiciesApiTestM1 - Release 0.1Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/39rollback test core update for core-common version2023-08-18T15:18:35ZNeelesh Thakurrollback test core update for core-common versionhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/6https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/6M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/37Update AWS version to match core2023-08-18T15:18:37ZMatt WiseUpdate AWS version to match coreM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/36Support for self-signed certificates for ElasticSearch connection(GONRG-776, ...2023-08-18T15:18:39ZRiabokon Stanislav(EPAM)[GCP]Support for self-signed certificates for ElasticSearch connection(GONRG-776, GONRG-539)## Type of change
- [ ] Bug Fix
- [X] Feature
https://community.opengroup.org/osdu/platform/system/home/-/issues/52
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider impl...## Type of change
- [ ] Bug Fix
- [X] Feature
https://community.opengroup.org/osdu/platform/system/home/-/issues/52
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does this introduce a breaking change?
- [YES]
## What is the current behavior?
Indexer does not support for self-signed certificates for ElasticSearch connection
## What is the new/expected behavior?
To use self-signed certificates for Elastic on non-production environments.
## Have you added/updated Unit Tests and Integration Tests?
-
## Any other useful information
### GCP Unify logic for credentials (GONRG-539)
# Description:
- Moved credentials classes into core-lib-gcp.
# How to test:
Business test cases.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:
Any comments to approvers hereM1 - Release 0.1Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/35Update .gitlab-ci.yml2023-08-18T15:18:40ZMatt WiseUpdate .gitlab-ci.ymlM1 - Release 0.1Matt WiseMatt Wisehttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/34Leveraging Azure Service Principle class to utilize from core lib azure2023-08-18T15:18:42Zsatish kumar narganiLeveraging Azure Service Principle class to utilize from core lib azureM1 - Release 0.1Christian LecknerDania Kodeih (Microsoft)Nandu MuralidharanChristian Lecknerhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/31GCP Unify logic for credentials (GONRG-539)2023-08-18T15:18:44ZRiabokon Stanislav(EPAM)[GCP]GCP Unify logic for credentials (GONRG-539)# Description:
- Moved credentials classes into core-lib-gcp.
# How to test:
Business test cases.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that s...# Description:
- Moved credentials classes into core-lib-gcp.
# How to test:
Business test cases.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:
Any comments to approvers hereM1 - Release 0.1Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/30Disabling AAD auth in indexer service2023-08-18T15:18:45ZKiran VeerapaneniDisabling AAD auth in indexer serviceAs Istio AuthN got enabled in R2 services,Disabling AAD authentication using Istio auth property. Added Istio auth Env variable and set to true in helm charts templateAs Istio AuthN got enabled in R2 services,Disabling AAD authentication using Istio auth property. Added Istio auth Env variable and set to true in helm charts templateM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/29multi-tenancy updates for Indexer in azure2023-08-18T15:18:47ZDuvelis Caraomulti-tenancy updates for Indexer in azureDraft merge requests can't be merged.Draft merge requests can't be merged.M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/28Enabled Slf4j and transaction logger in indexer-azure2023-08-18T15:18:48ZVibhuti Sharma [Microsoft]Enabled Slf4j and transaction logger in indexer-azure**Description**
* Enabled slf4j logger and transaction logger by adding the necessary commands in application.properties
* Added exclusions in `spring-boot-starter-security` dependency in pom.xml file to enable slf4j logger binding witho...**Description**
* Enabled slf4j logger and transaction logger by adding the necessary commands in application.properties
* Added exclusions in `spring-boot-starter-security` dependency in pom.xml file to enable slf4j logger binding without conflict
* Verified manually the reflection of test logs in `Traces` table in AppInsights
* Verified that existing integration tests passed
* Upgraded azure-core-lib version to 0.0.28 and made changes needed for the same
* \[YES\] Have you followed our code review [guidelines](https://github.com/microsoft/code-with-engineering-playbook/blob/master/pull-requests/code-reviews/readme.md)?
* \[YES\] Have you added an explanation of what your changes do and why you'd like us to include them?
* \[NO\] I have updated the documentation accordingly.
* \[NA\] I have added tests to cover my changes.
* \[YES\] All new and existing tests passed.
* \[YES\] My code follows the code style of this project.
* \[YES\] I ran lint checks locally prior to submissionM1 - Release 0.1Vibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/26use env var for code artifact domain. dynamically use current account as repo...2023-08-18T15:18:50ZMatt Wiseuse env var for code artifact domain. dynamically use current account as repo ownerM1 - Release 0.1