Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2022-11-23T15:54:25Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/412whitesource-update2022-11-23T15:54:25ZMadinabonu Alisherovawhitesource-updatewhitesource indexer updateswhitesource indexer updatesM15 - Release 0.18https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/235Wait for primary shards to be ready before start indexing2023-08-18T15:06:47ZNeelesh ThakurWait for primary shards to be ready before start indexingPlease refer to issue https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/43Please refer to issue https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/43M11 - Release 0.14https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/264Vuln fix lucene2023-08-18T15:06:26ZGokul NagareVuln fix luceneM10 - Release 0.13Shrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/710Vulnerbility fix on AWS side2024-03-19T17:11:10ZBruce JinVulnerbility fix on AWS sidethis will fix some AWS internal vulnerabilitiesthis will fix some AWS internal vulnerabilitiesM23 - Release 0.26Bruce JinBruce Jinhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/445Vulnerability fix ibm indexer service2023-03-30T08:07:20ZPintu GuptaVulnerability fix ibm indexer serviceFollowing CVE has been fix into this MR :
| cve | link |
|----------------|-------------------------------------------------|
| CVE-2022-42003 | https://nvd.nist.gov/vuln/detail/CVE-2...Following CVE has been fix into this MR :
| cve | link |
|----------------|-------------------------------------------------|
| CVE-2022-42003 | https://nvd.nist.gov/vuln/detail/CVE-2022-42003 |
| CVE-2022-42004 | https://nvd.nist.gov/vuln/detail/CVE-2022-42004 |
| CVE-2022-25857 | https://nvd.nist.gov/vuln/detail/CVE-2022-25857 |
| CVE-2022-42252 | https://nvd.nist.gov/vuln/detail/CVE-2022-42252 |M16 - Release 0.19Pintu GuptaPintu Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/634Vulnerability fix for Indexer Service2023-10-13T15:21:53ZKamalika SahaVulnerability fix for Indexer ServiceComponent Governance Link : https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_componentGovernance/oep-deployment-resources/alert/7626931?typeId=13890661&pipelinesTrackingFilter=0
Failed Pipeline link with CG name : http...Component Governance Link : https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_componentGovernance/oep-deployment-resources/alert/7626931?typeId=13890661&pipelinesTrackingFilter=0
Failed Pipeline link with CG name : https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_build/results?buildId=131859&view=logs&j=096205eb-4599-56da-affa-13246cc49eac
PBI link : https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_workitems/edit/31229
Build Success Screenshot : ![image](/uploads/acb7e0e86ff9a806985536a2c4a52469/image.png)
Service IT Pipeline Link : https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_build/results?buildId=132339&view=logs&j=18352b40-e004-552e-ac5b-401ff1a0f1fe&t=d1676cdd-0015-5643-b478-c4fe68a8be7d
Instance for which the Indexer image was pushed in : https://ms.portal.azure.com/#view/Microsoft_Azure_ContainerService/AksK8ResourceMenuBlade/~/overview-Deployment/aksClusterId/%2Fsubscriptions%2Fc99e2bf3-1777-412b-baba-d823676589c2%2FresourceGroups%2FCompute-rg-it1697053864288-jupwah%2Fproviders%2FMicrosoft.ContainerService%2FmanagedClusters%2Faks-bdftmqisqae2k/resource~/%7B%22kind%22%3A%22Deployment%22%2C%22metadata%22%3A%7B%22name%22%3A%22indexer%22%2C%22namespace%22%3A%22osdu-azure%22%2C%22uid%22%3A%2280490894-8de9-4474-88d0-162fbc485d53%22%7D%2C%22spec%22%3A%7B%22selector%22%3A%7B%22matchLabels%22%3A%7B%22app%22%3A%22indexer%22%7D%7D%7D%7DKamalika SahaKamalika Sahahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/510Vulnerability fix for IBM2023-07-25T07:11:22ZDevdatta SantraVulnerability fix for IBMVulnerability fix for IBM for tomcat and netty libs
| CVE-2022-41881 | https://nvd.nist.gov/vuln/detail/CVE-2022-41881 |
|----------------|-------------------------------------------------|
| CVE-2022-45143 | https://nvd.nist.gov/vuln/d...Vulnerability fix for IBM for tomcat and netty libs
| CVE-2022-41881 | https://nvd.nist.gov/vuln/detail/CVE-2022-41881 |
|----------------|-------------------------------------------------|
| CVE-2022-45143 | https://nvd.nist.gov/vuln/detail/CVE-2022-45143 |M17 - Release 0.20Devdatta SantraDevdatta Santrahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/563Vulnerability Fix2023-06-19T07:38:57ZKamalika SahaVulnerability FixKamalika SahaKamalika Sahahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/565vulnerabilities fixed2023-06-20T06:08:36ZAkshat Joshivulnerabilities fixedhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/567Vulnerabilite resolve2023-06-20T09:27:05ZAkshat JoshiVulnerabilite resolvehttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/381Versioning helm charts2022-09-09T20:26:55ZMarc Burnie [AWS]Versioning helm chartsM14 - Release 0.17Marc Burnie [AWS]Okoun-Ola Fabien HouetoGustavo UrdanetaMarc Burnie [AWS]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/150v3 feature file changes2021-04-29T15:04:54ZGokul Nagarev3 feature file changesPipeline is passing for IBM - https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/37609Pipeline is passing for IBM - https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/37609Anuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/200Using non SNAPSHOT version2023-08-18T15:07:09ZVibhuti Sharma [Microsoft]Using non SNAPSHOT versionM9 - Release 0.12Vibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/81Using Indexing Progress Topic2023-08-18T15:09:05ZKomal MakkarUsing Indexing Progress Topic## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have updated the documentation accordingly.
* [NA] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
The topic where indexing progress was being published was records changed. These changes assume that indexing status was constructed for this purpose.
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. --> [storage issue](https://community.opengroup.org/osdu/platform/system/storage/-/issues/43)
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [YES/NO] No
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->
**Correlation id sent as komakar2**
![image](/uploads/3558d5bb5bfa99979d006e511a03d66d/image.png)
**Indexing-progress has that as an active message with the same correlation-id**
![image](/uploads/9312a44f0474a158597d8940528a585d/image.png)M4 - Release 0.7Dania Kodeih (Microsoft)Hema Vishnu Pola [Microsoft]Kishore Battulaashley kelhamDania Kodeih (Microsoft)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/190Using AzureServicePrincipleTokenService in implementation for IServiceAccount...2023-08-18T15:07:21ZAbhishek PatilUsing AzureServicePrincipleTokenService in implementation for IServiceAccountJwtClientThe implementation of `IServiceAccountJwtClient` is having code which is already there in `AzureServicePrincipleTokenService` in core-lib-azure. Thus modifying the implementation of `IServiceAccountJwtClient` to use `AzureServicePrincipl...The implementation of `IServiceAccountJwtClient` is having code which is already there in `AzureServicePrincipleTokenService` in core-lib-azure. Thus modifying the implementation of `IServiceAccountJwtClient` to use `AzureServicePrincipleTokenService` for token generation logic.M9 - Release 0.12Abhishek PatilAbhishek Patilhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/245Users/amaverma/m8 log4j upgrade2021-12-16T01:45:49ZAman VermaUsers/amaverma/m8 log4j upgradeAll the versions <= 2.14 for slf4j have known security vulnerability. The recommendation is that we upgrade to `2.15.0` immediately.
Part of the #48 seriesAll the versions <= 2.14 for slf4j have known security vulnerability. The recommendation is that we upgrade to `2.15.0` immediately.
Part of the #48 seriesAman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/199use redis cache for azure2023-08-18T15:07:11ZNeelesh Thakuruse redis cache for azureRedis cache implementation for Azure Indexer service resources. This will help in:
- Minimize calls to Schema service & use the same flatten schema by all service instances
- Correctly keep track of index status
- Service Principle JWT c...Redis cache implementation for Azure Indexer service resources. This will help in:
- Minimize calls to Schema service & use the same flatten schema by all service instances
- Correctly keep track of index status
- Service Principle JWT creationM9 - Release 0.12https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/307use index delete api for IT post clean-up2023-08-18T15:05:54ZNeelesh Thakuruse index delete api for IT post clean-upUse index delete api for IT post clean-up, it will properly clear cache on index deletion, reducing IT flakiness. It also removes usage of deprecated index-cleanup endpoint in GCP.Use index delete api for IT post clean-up, it will properly clear cache on index deletion, reducing IT flakiness. It also removes usage of deprecated index-cleanup endpoint in GCP.M12 - Release 0.15https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/488use Index API to update the record2023-03-03T14:33:50ZMykyta Savchukuse Index API to update the recordIndexer uses Update API (UpdateRequest) to update the record with partial update, which means that fields that are not included in the request won't be updated/deleted, but is expected to use full replacement, where field that are not in...Indexer uses Update API (UpdateRequest) to update the record with partial update, which means that fields that are not included in the request won't be updated/deleted, but is expected to use full replacement, where field that are not included are deleted, since storage works this way with PUT endpoint. Because of this it is not possible to delete tags and some data fields. To fix this it is possible to use Index API (IndexRequest) for both insert and update.
Docs:
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docs-update.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docs-index_.htmlM17 - Release 0.20Chad LeongMykyta SavchukChad Leonghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/684Use full URL instead of relative path2024-02-09T14:24:31ZLawrence ChanUse full URL instead of relative path* Issue Reference: [#68](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/issues/68)
* Added configuration `api.server.fullUrl.enabled` to enable full server url in OpenAPI swagger
* Currently only in Azure i...* Issue Reference: [#68](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/issues/68)
* Added configuration `api.server.fullUrl.enabled` to enable full server url in OpenAPI swagger
* Currently only in Azure it is enabled. For Other \[CSP/Common Core\] there is no change.
### Configuration Details
* `api.server.fullUrl.enabled=true` It will generate full server url in the OpenAPI swagger
* `api.server.fullUrl.enabled=false` It will generate only the contextPath
* Reference: https://springdoc.org/faq.html#_how_is_server_url_generatedM23 - Release 0.26