Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2024-03-04T12:23:17Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/694Support parsing own output format for datetimes2024-03-04T12:23:17ZStanisław BienieckiSupport parsing own output format for datetimesResolves: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/138
Currently indexer doesn't accept datetime format which formatter produces as an output. This is causing for example that you cannot simply copy ...Resolves: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/138
Currently indexer doesn't accept datetime format which formatter produces as an output. This is causing for example that you cannot simply copy datetimes between resources.M23 - Release 0.26https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/713Fix the issue that augmenter throws null pointer exception when casting the r...2024-02-26T14:54:23ZZhibin MaiFix the issue that augmenter throws null pointer exception when casting the related object ids retrieved from the GeoContextThe bug was introduced in M21. For the detail of the issue, please refer to issue [151](https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/151).The bug was introduced in M21. For the detail of the issue, please refer to issue [151](https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/151).M23 - Release 0.26Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/693Fix parsing string arrays2024-02-08T19:27:30ZStanisław BienieckiFix parsing string arraysCloses https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/137
Hotfix for parsing string arrays, as previous fix in handling string introduced error in parsing string arraysCloses https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/137
Hotfix for parsing string arrays, as previous fix in handling string introduced error in parsing string arraysM22 - Release 0.25https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/699resiliency fixes on cache miss2024-01-27T14:42:28ZNeelesh Thakurresiliency fixes on cache missresiliency fixes on cache miss
- retry fetching schema from schema service if there is a cache miss, otherwise it will incorrectly index the record.
- skip populating VirtualProperties if there a cache, otherwise it throws NullPointerExc...resiliency fixes on cache miss
- retry fetching schema from schema service if there is a cache miss, otherwise it will incorrectly index the record.
- skip populating VirtualProperties if there a cache, otherwise it throws NullPointerException.
Provide audit log entry for partially successful indexed record. Currently we are populating only for successful or failed request only.M23 - Release 0.26https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/703Cherry-pick 'Fix parsing string arrays' into release/0.252024-01-24T11:49:37ZChad LeongCherry-pick 'Fix parsing string arrays' into release/0.25**Original MR**: !693
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !693
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/new?ref=cherry-pick-for-693)M22 - Release 0.25David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/673Cherry-pick 'Resolve the types of the augmented properties recursively' into ...2023-12-16T10:24:04ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Resolve the types of the augmented properties recursively' into release/0.25**Original MR**: !669
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !669
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/new?ref=cherry-pick-for-669)M22 - Release 0.25David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/669Resolve the types of the augmented properties recursively2023-12-15T20:06:39ZZhibin MaiResolve the types of the augmented properties recursivelyIssue [120](https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/120)
The fix can resolve schema of the augmented properties recursively. It also has logic to prevent infinite loop when circular references exist...Issue [120](https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/120)
The fix can resolve schema of the augmented properties recursively. It also has logic to prevent infinite loop when circular references exist. In the unit test examples:
- It travels 3 levels of relationships to get the schema of the augmented properties
- `Wellbore` augments `data.Curves[].Mnemonic` as augmented property `WellLogs` from kind `WellLog` while `WellLog` augments several properties from `Wellbore`.M22 - Release 0.25Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/630Change the default setting of trackTotalCount to false in SearchRequest2023-11-27T20:15:12ZZhibin MaiChange the default setting of trackTotalCount to false in SearchRequestChange the default setting of trackTotalCount to false in SearchRequest. Setting trackTotalCount to true could increase the overhead on ElasticSearch and affect the performance.Change the default setting of trackTotalCount to false in SearchRequest. Setting trackTotalCount to true could increase the overhead on ElasticSearch and affect the performance.M21 - Release 0.24Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/628resolve the bug of "/reindex/record API returns 500 when there is no not-foun...2023-10-13T11:53:33ZMingyang Zhuresolve the bug of "/reindex/record API returns 500 when there is no not-found records"fix the bug and add test cases to cover the scenariofix the bug and add test cases to cover the scenarioM21 - Release 0.24Mingyang ZhuMingyang Zhuhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/624Cherry-pick 'fix normalizedKind tag index bug' into release/0.232023-09-29T06:46:36ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'fix normalizedKind tag index bug' into release/0.23**Original MR**: !621
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !621
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/indexer-service/-/pipelines/new?ref=cherry-pick-for-621)M20 - Release 0.23David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/617To avoid conflict call, rename classes to make IBM indexer service up.2023-09-28T21:18:12ZAshwani PandeyTo avoid conflict call, rename classes to make IBM indexer service up.To avoid conflict call, rename classes to make IBM indexer service up.
See merge request osdu/platform/system/indexer-service!616To avoid conflict call, rename classes to make IBM indexer service up.
See merge request osdu/platform/system/indexer-service!616M20 - Release 0.23Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/621fix normalizedKind tag index bug2023-09-28T21:16:08ZMingyang Zhufix normalizedKind tag index bugfix normalizedKind tag index bug. The original code won't index the normalizedKind tag if the storage record has tags
original ADR: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/78
bug item: https://commun...fix normalizedKind tag index bug. The original code won't index the normalizedKind tag if the storage record has tags
original ADR: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/78
bug item: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/115M20 - Release 0.23Mingyang ZhuMingyang Zhuhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/616Ibm issue fix2023-09-21T07:27:16ZAshwani PandeyIbm issue fixM20 - Release 0.23Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/607Updated schema cache name (7657)2023-08-25T14:14:46ZYurii Ruban [EPAM / GCP]Updated schema cache name (7657)# Description:
Fix application context.
# How to test:
Check int tests
# Changes include:
- [ ] Bugfix (a non-breaking change that solves an issue).
# Changes in:
- [ ] GCP# Description:
Fix application context.
# How to test:
Check int tests
# Changes include:
- [ ] Bugfix (a non-breaking change that solves an issue).
# Changes in:
- [ ] GCPM20 - Release 0.23Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/13fix: don't log stack traces for missing record properties2023-08-18T15:19:04ZBrandt Bealfix: don't log stack traces for missing record propertiesWhen indexer looks for properties on a storage record and a property can't be found an exception is thrown which is logged. Since the log statement includes the original exception the stack trace is unwound creating a performance issue (...When indexer looks for properties on a storage record and a property can't be found an exception is thrown which is logged. Since the log statement includes the original exception the stack trace is unwound creating a performance issue (cpu/network) which essentially kills underpowered containers. The exception stack trace doesn't provide any additional information and by removing it fixes the performance issue for current data ingestion workloads.
`o.o.o.c.common.logging.DefaultLogWriter : indexer.app: record-id: opendes:doc:b55f7ee408354a7b9b48cd2b443fbf16 | error fetching property: Data.IndividualTypeProperties.Curves.InterpreterName | error: Unknown property 'InterpreterName' on class 'class java.util.ArrayList' {correlation-id=16b8f17d-3ded-462d-9c14-83204bf45fb9, data-partition-id=opendes}`M1 - Release 0.1Dania Kodeih (Microsoft)Brandt BealDaniel SchollDmitriy RudkoAlan BrazDania Kodeih (Microsoft)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/22Fix vulnerabilities in dependencies found in result of scqp checking2023-08-18T15:18:55ZYauheni LesnikauFix vulnerabilities in dependencies found in result of scqp checkingFix next vulnerabilities in dependencies found in result of scqp checking for core and azure modules
```
CORE:
| Library | ...Fix next vulnerabilities in dependencies found in result of scqp checking for core and azure modules
```
CORE:
| Library | |
|----------------------------------------|----------------------------------------------------------------------------
|
| jackson-databind-2.9.9.jar |
|
| hibernate-validator-6.0.17.Final.jar | |
| snakeyaml-1.23.jar | |
| commons-codec-1.11.jar | |
| tomcat-embed-core-9.0.21.jar | |
| netty-codec-4.1.38.Final.jar | | | commons-beanutils-1.9.3.jar |
AZURE:
| Library | |
|--------------------------------------|----------------------------------------------------------------------------
|
| jackson-databind-2.9.9.jar |
|
| netty-codec-http-4.1.38.Final.jar | |
| reactor-netty-0.8.10.RELEASE.jar | |
| netty-codec-http2-4.1.38.Final.jar | |
| hibernate-validator-6.0.12.Final.jar | |
| snakeyaml-1.23.jar | |
| commons-codec-1.11.jar | |
| tomcat-embed-core-9.0.22.jar | |
| netty-codec-4.1.38.Final.jar | |
| log4j-core-2.11.2.jar | | | bcprov-jdk15on-1.60.jar |
|
| commons-beanutils-1.9.3.jar |
|
| nimbus-jose-jwt-6.0.2.jar |
|
|jackson-mapper-asl-1.9.13.jar |
```M1 - Release 0.1ethiraj krishnamanaiduNitin-slbNeelesh ThakurPavel BachylaAliaksei DarafeyeuYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/25increase spring boot version;2023-08-18T15:18:52ZYauheni Lesnikauincrease spring boot version;Updating spring boot version from 2.1.7.RELEASE to 2.1.16.RELEASEUpdating spring boot version from 2.1.7.RELEASE to 2.1.16.RELEASEM1 - Release 0.1ethiraj krishnamanaiduNitin-slbNeelesh ThakurSherman YangAliaksei DarafeyeuYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/53fix whistesource issues for the indexer-core and indexer-azure;2023-08-18T15:17:29ZYauheni Lesnikaufix whistesource issues for the indexer-core and indexer-azure;Fix whistesource issues for the indexer-core and indexer-azure;
Spring-boot version increased to the latest one of the 2.1.x series.
com.fasterxml.woodstox:woodstox-core - increased to the latest oneFix whistesource issues for the indexer-core and indexer-azure;
Spring-boot version increased to the latest one of the 2.1.x series.
com.fasterxml.woodstox:woodstox-core - increased to the latest oneM1 - Release 0.1ethiraj krishnamanaiduNeelesh ThakurDuvelis CaraoYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/77fix unexpected conversion from int to double2023-08-18T15:09:10ZNeelesh Thakurfix unexpected conversion from int to doubleFix unexpected conversion from int to double caused by Gson lib. Use Jackson lib to deserialize storage service response instead.Fix unexpected conversion from int to double caused by Gson lib. Use Jackson lib to deserialize storage service response instead.M6 - Release 0.9https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/83Fix Int tests failing due to new storage record ID validation rules2023-08-18T15:09:03ZMatt WiseFix Int tests failing due to new storage record ID validation rulesM3 - Release 0.5ethiraj krishnamanaiduMatt Wiseethiraj krishnamanaidu