Indexer Queue merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests2023-08-18T15:15:02Zhttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/74Add var for indexer queue identifier [GONRG-2579]2023-08-18T15:15:02ZMikhail Piatliou (EPAM)Add var for indexer queue identifier [GONRG-2579]M7 - Release 0.10Oleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/73Publish helm charts for indexer-queue [GONRG-2483]2023-08-18T15:15:04ZMikhail Piatliou (EPAM)Publish helm charts for indexer-queue [GONRG-2483]M7 - Release 0.10Oleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/72Fix containerize step for gitlab [GONRG-2483]2023-08-18T15:15:05ZMikhail Piatliou (EPAM)Fix containerize step for gitlab [GONRG-2483]M7 - Release 0.10Oleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/71Making Enqueue Azure a Spring Boot Application2023-08-18T15:15:07ZVibhuti Sharma [Microsoft]Making Enqueue Azure a Spring Boot ApplicationThis merge request addresses issue https://community.opengroup.org/osdu/platform/system/indexer-queue/-/issues/6. It contains code changes and `devops/azure/charts` changes to deploy indexer-queue as a spring boot application.
Performanc...This merge request addresses issue https://community.opengroup.org/osdu/platform/system/indexer-queue/-/issues/6. It contains code changes and `devops/azure/charts` changes to deploy indexer-queue as a spring boot application.
Performance Testing results can be found here - https://dev.azure.com/osdu-demo/osdu-perf/_wiki/wikis/osdu-perf.wiki/398/Run-4
Implementation Details
* Uses the Azure Client Library for Java to receive messages from service bus topics
* Uses Partition Service to fetch service bus connection information for each data partition
* Implements DpsHeaders as Thread Scoped beans instead of request scoped beans. This is because indexer queue is a worker service with no public APIs so there is no context of request scope.
* Deleted file `log4J2.xml` as it is now directly pulled from core-lib-azure.
Testing
* Perf tested.
* Integration and unit tested.
* Pipelines green.M8 - Release 0.11Vibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/70Add node selector2021-08-05T10:39:45ZDzmitry_Paulouski (slb)Add node selector## All Submissions:
-------------------------------------
* [YES] Have you followed our code review [guidelines](https://github.com/microsoft/code-with-engineering-playbook/blob/master/pull-requests/code-reviews/readme.md)?
* [YES] Have ...## All Submissions:
-------------------------------------
* [YES] Have you followed our code review [guidelines](https://github.com/microsoft/code-with-engineering-playbook/blob/master/pull-requests/code-reviews/readme.md)?
* [YES] Have you added an explanation of what your changes do and why you'd like us to include them?
* [NO] I have updated the documentation accordingly.
* [NA] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the current behavior?
-------------------------------------
Currently, we have only one node pool for services and system workloads.
## What is the new behavior?
-------------------------------------
We are going to move workloads to a new node pool with enabled autoscaling. Here you can find the details:
[Issue link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/167)
## Does this introduce a breaking change?
-------------------------------------
- [NO]
This change has a strong dependency on the following MR and should be merged **only** after it:
https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/324https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/69Added PARTITION_API variable2023-08-18T15:15:09ZOleksii Tsyganov (EPAM)Added PARTITION_API variableM6 - Release 0.9Oleksii Tsyganov (EPAM)Oleksii Tsyganov (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/68Partion service as tenantinfo provider (GONRG-2257)2023-08-18T15:15:10ZAnastasiia GelmutPartion service as tenantinfo provider (GONRG-2257)# Description:
Added optional service that replace TenantInfoFactory with implementation that utilize Partition Service<br/>
# How to test:
Functionality was tested with Partition service, Indexer Queue service successfully consume Ten...# Description:
Added optional service that replace TenantInfoFactory with implementation that utilize Partition Service<br/>
# How to test:
Functionality was tested with Partition service, Indexer Queue service successfully consume TenantInfo from Partition Service
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [x] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [x] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:
Dependencies:<br/>
https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp/-/merge_requests/12 <br/>
https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/73 <br/>M6 - Release 0.9Riabokon Stanislav(EPAM)[GCP]Rostislav Dublin (EPAM)Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/67aws-multitenant-v22023-08-18T15:15:12ZRucha Deshpandeaws-multitenant-v2M6 - Release 0.9Rucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/66Switching the dependencies to release versions2023-08-18T15:15:14ZDavid Diederichd.diederich@opengroup.orgSwitching the dependencies to release versionsThis changes the library dependencies to use released versions of the core libraries.
They were previously depending on SNAPSHOT versions, which are less stable versions.
More importantly, the SNAPSHOT versions are periodically purged fr...This changes the library dependencies to use released versions of the core libraries.
They were previously depending on SNAPSHOT versions, which are less stable versions.
More importantly, the SNAPSHOT versions are periodically purged from the system to save disk space -- this happened recently.
Since these libraries no longer exist on community, building becomes difficult.
This MR moves those dependencies to a release version, which is better going forward and allows FOSSA to do the build and get good dependency information.
I assert that there are no substantial changes between the SNAPSHOT version I moved from and the latest release version that I moved to.
It's difficult to know which commit the SNAPSHOT dependency linked to, since it moved many times, but here are the differences from the last time the SNAPSHOT dependency was listed and the one commit that has the release version (0.7.0).
All of these changes were from me, updating versions and references as part of the release process.
- [GCP Differences from v0.6.1-SNAPSHOT to 0.7.0](https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp/-/compare/ff52818d929b7a32e491b75743285026c4c0a9b4...v0.7.0)
- [IBM Differences from v0.3.8-SNAPSHOT to 0.7.0](https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm/-/compare/42d057eeee8f0299e7f90f1158c24f7fc0187dbc...v0.7.0)
Separately, since I was working with FOSSA, I updated the configuration file and the corresponding NOTICE changes resulting from the new module.M5 - Release 0.8David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/65update notice file2023-08-18T15:15:15ZBill Wangupdate notice fileM5 - Release 0.8Bill WangBill Wanghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/64Aws integration -- Add ReIndex Async Processing2023-08-18T15:15:17ZBill WangAws integration -- Add ReIndex Async ProcessingAsync Processing of ReIndex Pagination RequestsAsync Processing of ReIndex Pagination RequestsM5 - Release 0.8Bill WangBill Wanghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/63Draft: Consistent naming for Container and Spring App names2021-03-10T15:01:10ZKrishna Nikhil VedurumudiDraft: Consistent naming for Container and Spring App names## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provi...## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
The Cloud Role name used for the service is `indexer-queue` whereas the Spring Application Name was not set.
Due to which the templatized dashboards that we have created have started to fail and not give results for indexer service and indexer queue.
## What is the new/expected behavior?
Rename `springboot-application-name`
## Have you added/updated Unit Tests and Integration Tests?
N/A
## Any other useful informationhttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/62Aws integration2023-08-18T15:15:19ZMatt WiseAws integrationM4 - Release 0.7Matt WiseBill WangMatt Wisehttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/61(GONRG-1765) GCP new log implementation2023-08-18T15:15:21ZIgor Filippov (EPAM)(GONRG-1765) GCP new log implementation## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IB...## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## Have you added/updated Unit Tests and Integration Tests?
- [NO]M4 - Release 0.7Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/60AWS Updates2023-08-18T15:15:23ZSpencer Suttonsuttonsp@amazon.comAWS UpdatesM4 - Release 0.7ethiraj krishnamanaiduDania Kodeih (Microsoft)Wladmir FrazaoJoeDmitriy Rudkoethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/59(GONRG-1765) GCP logging impl2023-08-18T15:15:25ZIgor Filippov (EPAM)(GONRG-1765) GCP logging impl## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IB...## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## Have you added/updated Unit Tests and Integration Tests?
- [YES]M4 - Release 0.7Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/58Cherry pick "GCP release/0.5 - Fix containerize CICD step" into release/0.52021-02-11T20:47:02ZDavid Diederichd.diederich@opengroup.orgCherry pick "GCP release/0.5 - Fix containerize CICD step" into release/0.5This merges !57 to the release/0.5 branch to fix the osdu-gcp-containerize step for non-SNAPSHOT builds.This merges !57 to the release/0.5 branch to fix the osdu-gcp-containerize step for non-SNAPSHOT builds.David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/57GCP release/0.5 - Fix containerize CICD step2023-08-18T15:15:26ZDmitriy RudkoGCP release/0.5 - Fix containerize CICD stepM3 - Release 0.5Dmitriy RudkoDmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/56WIP: Fix GCP for release/0.52021-02-11T17:02:10ZDmitriy RudkoWIP: Fix GCP for release/0.5https://community.opengroup.org/osdu/platform/system/indexer-queue/-/merge_requests/55Cherry picks for 0.52021-02-11T05:51:07ZDavid Diederichd.diederich@opengroup.orgCherry picks for 0.5This cherry-picks MR !53 and !54 from master into the release branch, without adding any extra commits that aren't necessary.This cherry-picks MR !53 and !54 from master into the release branch, without adding any extra commits that aren't necessary.David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.org