Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2023-07-25T19:59:14Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/587enhance logging to include correlation-id and record-id2023-07-25T19:59:14ZLarissa Pereiraenhance logging to include correlation-id and record-idAdding more information like correlation-id and record-id to the logs to help better investigate issues in future. While debugging a certain issue with indexing, we noticed that indexer service logs were not correlated with storage and i...Adding more information like correlation-id and record-id to the logs to help better investigate issues in future. While debugging a certain issue with indexing, we noticed that indexer service logs were not correlated with storage and indexer-queue logs so it was hard to track down all the related logs. Also, it would help to capture the record-id in the logs for which the indexing process started, so that issues can be traced to the specific record-id.M20 - Release 0.23Chad LeongLarissa PereiraChad Leonghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/716Enhance the Augmenter documentation2024-03-08T16:48:00ZZhibin MaiEnhance the Augmenter documentationUpdate Augmenter Document from the following aspects:
* Update introduction section
* Update use cases
* Remove all the `data.` prefix in ValuePath
* Make sure the `Policy` and `RelationshipDirection` are set correctlyUpdate Augmenter Document from the following aspects:
* Update introduction section
* Update use cases
* Remove all the `data.` prefix in ValuePath
* Make sure the `Policy` and `RelationshipDirection` are set correctlyM23 - Release 0.26Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/155Entitlements V22023-08-18T15:08:03ZRucha DeshpandeEntitlements V2commit 934ae6f0
Author: Rucha Deshpande <deshruch@amazon.com>
Date: Wed May 05 2021 17:05:58 GMT-0500 (Central Daylight Time)
update entitlements domain
commit 63173bf4
Author: Rucha Deshpande <deshruch@amazon.com>
Date: Wed ...commit 934ae6f0
Author: Rucha Deshpande <deshruch@amazon.com>
Date: Wed May 05 2021 17:05:58 GMT-0500 (Central Daylight Time)
update entitlements domain
commit 63173bf4
Author: Rucha Deshpande <deshruch@amazon.com>
Date: Wed May 05 2021 11:42:30 GMT-0500 (Central Daylight Time)
use Entitlements V2M6 - Release 0.9Rucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/115entitlements v2 cutover2023-08-18T15:08:36ZMingyang Zhuentitlements v2 cutoverM5 - Release 0.8Mingyang ZhuMingyang Zhuhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/274Explicit json-smart dependency2022-09-16T02:12:25ZAlok JoshiExplicit json-smart dependencyUpgrade json-smart dependency to 2.4.7Upgrade json-smart dependency to 2.4.7M10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/63Feat/aws impl2022-09-16T09:20:35ZRucha DeshpandeFeat/aws implIntegration test fixes for index-cleanup api.
Added a non-breaking change: Add CleanupIndiciesSteps constructor that accepts an instance of ElasticUtilsIntegration test fixes for index-cleanup api.
Added a non-breaking change: Add CleanupIndiciesSteps constructor that accepts an instance of ElasticUtilsM1 - Release 0.1ethiraj krishnamanaiduJoeRucha Deshpandeethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/70Feat/aws schema service int2023-08-18T15:09:22ZRucha DeshpandeFeat/aws schema service intM1 - Release 0.1Rucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/708fix aws entitlement url2024-02-08T15:46:32ZYunhua Koglinfix aws entitlement urlM23 - Release 0.26Yunhua KoglinYunhua Koglinhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/629fix azure io.netty, vulnerability and exclude document db2023-10-12T06:10:36ZVidyaDharani Lokamfix azure io.netty, vulnerability and exclude document db# Change details
* excluded unused dependency `documentdb-bulkexecutor`
* upgrade `io.netty:netty-bom` version to `4.1.98.Final`
# Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBM# Change details
* excluded unused dependency `documentdb-bulkexecutor`
* upgrade `io.netty:netty-bom` version to `4.1.98.Final`
# Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBMM21 - Release 0.24VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/182fix bug to allow indexing of nested geopoint2023-08-18T15:07:30ZNeelesh Thakurfix bug to allow indexing of nested geopointAddresses issue #29Addresses issue #29M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/313Fix coordinates deserialization errors2022-09-15T10:17:20ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comFix coordinates deserialization errors# Description:
Issue: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/63
There is a bug related to the deserialization of such geodata as `MultiLineString`, `MultiPoint`, `LineString`, `MultiPolygon` whenev...# Description:
Issue: https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/63
There is a bug related to the deserialization of such geodata as `MultiLineString`, `MultiPoint`, `LineString`, `MultiPolygon` whenever such type of geodata contains `altitude`, deserialization does not work correctly in various ways. As a fix, we suggest deserializing coordinates for those types explicitly, taking into account the level of nesting of those coordinates.
# How to test:
Functionality was tested via unit tests, and record index requests, after fixing records that contain geodata `MultiLineString`, `MultiPoint`, `LineString`, `MultiPolygon` with `altitude' becomes searchable:
~~~
curl --location --request POST 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/search/v2/query' \
--header 'data-partition-id: odesprod' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer' \
--data-raw '{
"kind": "odesprod:wks:master-data--Wellbore:1.0.0",
"query": "id:\"odesprod:master-data--Wellbore:5May3Dimplicit\""
}'
~~~
~~~
{
"results": [
{
"data": {
"GeographicBottomHoleLocation.SpatialParameterTypeID": null,
"GeographicBottomHoleLocation.CoordinateQualityCheckPerformedBy": null,
"ResourceLifecycleStatus": null,
"TargetFormation": null,
"DefaultVerticalMeasurementID": "Measured_From",
"SpatialLocation.Wgs84Coordinates": {
"geometries": [
{
"coordinates": [
[
34.51305348816962,
0.035298719063461145,
7.0
],
[
34.513062650895186,
0.030790856372375844,
7.0
],
[
34.513071785973196,
0.0262829935696415,
7.0
]
],
"type": "linestring"
}
],
"type": "geometrycollection"
},
....
~~~
# Changes include:
- [x] Bugfix (a non-breaking change that solves an issue).
# Changes in:
- [x] Common code
# Dev Checklist:
- [x] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.M12 - Release 0.15Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/675Fix dependency upgrades for Azure2023-12-18T13:58:42ZThulasi Dass SubramanianFix dependency upgrades for Azure- Fix dependency upgrades for Azure
- upgrade jackson-bom version- Fix dependency upgrades for Azure
- upgrade jackson-bom versionThulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/640fix: disabling test cases for indexer extended2023-10-18T22:54:21ZKiran Veerapanenifix: disabling test cases for indexer extendedKiran VeerapaneniKiran Veerapanenihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/546fix documentation as details block was hiding rest of the documentation2023-05-15T15:23:33ZNeelesh Thakurfix documentation as details block was hiding rest of the documentation- fix documentation as details block was hiding rest of the documentation
- remove r3 release specific documentation- fix documentation as details block was hiding rest of the documentation
- remove r3 release specific documentationM18 - Release 0.21https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/13fix: don't log stack traces for missing record properties2023-08-18T15:19:04ZBrandt Bealfix: don't log stack traces for missing record propertiesWhen indexer looks for properties on a storage record and a property can't be found an exception is thrown which is logged. Since the log statement includes the original exception the stack trace is unwound creating a performance issue (...When indexer looks for properties on a storage record and a property can't be found an exception is thrown which is logged. Since the log statement includes the original exception the stack trace is unwound creating a performance issue (cpu/network) which essentially kills underpowered containers. The exception stack trace doesn't provide any additional information and by removing it fixes the performance issue for current data ingestion workloads.
`o.o.o.c.common.logging.DefaultLogWriter : indexer.app: record-id: opendes:doc:b55f7ee408354a7b9b48cd2b443fbf16 | error fetching property: Data.IndividualTypeProperties.Curves.InterpreterName | error: Unknown property 'InterpreterName' on class 'class java.util.ArrayList' {correlation-id=16b8f17d-3ded-462d-9c14-83204bf45fb9, data-partition-id=opendes}`M1 - Release 0.1Dania Kodeih (Microsoft)Brandt BealDaniel SchollDmitriy RudkoAlan BrazDania Kodeih (Microsoft)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/240Fix for istio dns host2023-08-18T15:06:42ZAnkur RawatFix for istio dns hostEmpty istioDnsHost is not allowed. Fix for the same.Empty istioDnsHost is not allowed. Fix for the same.M10 - Release 0.13Ankur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/212fix geojson parsing issues - only LAST geometry for "type": "AnyCrsGeometryCo...2023-08-18T15:06:59ZNeelesh Thakurfix geojson parsing issues - only LAST geometry for "type": "AnyCrsGeometryCollection" is indexed & throws exception on three coordinatesAddresses issue: #37 & #38Addresses issue: #37 & #38M9 - Release 0.12https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/637fix: igonring experimental test cases2023-10-17T15:40:44ZKiran Veerapanenifix: igonring experimental test casesKiran VeerapaneniKiran Veerapanenihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/141Fixing Azure On Release Branch2021-03-29T18:12:19ZJasonFixing Azure On Release BranchAzure seems to need updated os-core-common version on release branch. This change makes Azure pass test pipeline.Azure seems to need updated os-core-common version on release branch. This change makes Azure pass test pipeline.David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/593Fixing concurrency issues with getIndexMappingFromRecordSchema()2023-08-05T06:56:09ZSabarish K R EFixing concurrency issues with getIndexMappingFromRecordSchema()test case demonstrating a thread-unsafe operation of IndexerMappingServiceImpl.getIndexMappingFromRecordSchema(..) and also the fix for it.
When concurrent indexing requests for kinds whose index hasn't been created yet arrive, There ar...test case demonstrating a thread-unsafe operation of IndexerMappingServiceImpl.getIndexMappingFromRecordSchema(..) and also the fix for it.
When concurrent indexing requests for kinds whose index hasn't been created yet arrive, There are concurrent calls to getIndexMappingFromRecordSchema. There is a threading issue in TypeMapper.java - It gives the same objects to multiple threads, which result in the objects being corrupted with another object's value. This PR fixes the problem by ensuring every function call to the TypeMapper methods return new objects and not static pre-created objects.