Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2020-06-08T20:19:05Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/6Trusted ibm2020-06-08T20:19:05ZAlan BrazTrusted ibmDavid Diederichd.diederich@opengroup.orgethiraj krishnamanaiduDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/12fix: don't log stack traces for missing record properties2020-07-08T17:21:19ZBrandt Bealfix: don't log stack traces for missing record propertiesWhen indexer looks for properties on a storage record and a property can't be found an exception is thrown which is logged. Since the log statement includes the original exception the stack trace is unwound creating a performance issue ...When indexer looks for properties on a storage record and a property can't be found an exception is thrown which is logged. Since the log statement includes the original exception the stack trace is unwound creating a performance issue (cpu/network) which essentially kills underpowered containers. The exception stack trace doesn't provide any additional information and by removing it fixes the performance issue for current data ingestion workloads.
```o.o.o.c.common.logging.DefaultLogWriter : indexer.app: record-id: opendes:doc:b55f7ee408354a7b9b48cd2b443fbf16 | error fetching property: Data.IndividualTypeProperties.Curves.InterpreterName | error: Unknown property 'InterpreterName' on class 'class java.util.ArrayList' {correlation-id=16b8f17d-3ded-462d-9c14-83204bf45fb9, data-partition-id=opendes}```https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/16Port the latest clean-up cron changes2020-09-15T07:14:53ZSviatoslav NekhaienkoPort the latest clean-up cron changes* What is the change? This change is to port the latest clean-up cron changes from GA
* What does it impact? Adds a new endpoint to run a clean-up job for all providers.
* How do I reproduce or test the change? You can trigger GET reque...* What is the change? This change is to port the latest clean-up cron changes from GA
* What does it impact? Adds a new endpoint to run a clean-up job for all providers.
* How do I reproduce or test the change? You can trigger GET request via /_dps/task-handlers/index-cleanup, unit tests were added as well
* Other
Following properties(env variables) must be set
CRON_INDEX_CLEANUP_PATTERN
CRON_INDEX_CLEANUP_TENANTS
GAE_SERVICENeelesh ThakurSviatoslav NekhaienkoNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/18Aws integration2020-08-06T14:31:56ZMatt WiseAws integrationhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/20WIP: GCP Unify logic for credentials & support self signed cert for elastic (...2020-09-15T07:45:30ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comWIP: GCP Unify logic for credentials & support self signed cert for elastic (GONRG-539)# Description:
- Moved credentials classes into core-lib-gcp.
- Added a new 'ElasticClientHandlerGcp' to support self-signed certificates.
- Added a new env variables 'SECURITY_HTTPS_CERTIFICATE_TRUST' into app.yaml.
- Added a property '...# Description:
- Moved credentials classes into core-lib-gcp.
- Added a new 'ElasticClientHandlerGcp' to support self-signed certificates.
- Added a new env variables 'SECURITY_HTTPS_CERTIFICATE_TRUST' into app.yaml.
- Added a property 'security.https.certificate.trust' into application-*.properties.
# How to test:
Turn on https with a custom self-signed cert for elastic. SECURITY_HTTPS_CERTIFICATE_TRUST = true
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [x] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:
Any comments to approvers hereDmitriy RudkoDmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/32WIP: indexer (GONRG-549)2020-09-28T14:25:40ZMykola Zamkovyi (EPAM)WIP: indexer (GONRG-549)Oleksandr Kosse (EPAM)Mykola Zamkovyi (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/38Fix indexer-core version in GCP and IBM provider POMs2020-10-07T19:40:51ZAlok JoshiFix indexer-core version in GCP and IBM provider POMsGCP and IBM provider POMs are using an outdated version of indexer-core as a dependency. This change will update that to use the current version.GCP and IBM provider POMs are using an outdated version of indexer-core as a dependency. This change will update that to use the current version.Neelesh ThakurNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/40ibm sync with develop2020-10-16T09:58:13ZGokul Nagareibm sync with developAnuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/27IBM impl index enhancement2020-10-19T11:50:21ZAnuj GuptaIBM impl index enhancementIBM Indexer enhancements for indexerIBM Indexer enhancements for indexerAnuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/44WIP: Add gcp artifact2020-10-19T14:01:21ZOleksandr Kosse (EPAM)WIP: Add gcp artifacthttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/82Fix failing int tests due to new Storage Record ID validation rules2021-02-01T18:04:13ZMatt WiseFix failing int tests due to new Storage Record ID validation ruleshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/84Trusted fixing test case schemas2021-02-03T19:20:31ZJasonTrusted fixing test case schemashttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/93Search tagging2021-02-25T16:43:01ZSviatoslav NekhaienkoSearch tagging- Add tags support in the search results- Add tags support in the search resultsSviatoslav NekhaienkoSviatoslav Nekhaienkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/95Draft: Aws integration2023-09-13T23:03:23ZMatt WiseDraft: Aws integrationethiraj krishnamanaiduSpencer Suttonsuttonsp@amazon.comMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/98Removed elastic 6.6.2 from int tests ibm2021-03-01T12:49:27ZRiabokon Stanislav(EPAM)[GCP]Removed elastic 6.6.2 from int tests ibmRemoved elastic 6.6.2 from int tests ibm
## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?...Removed elastic 6.6.2 from int tests ibm
## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [X] IBM
## Does this introduce a breaking change?
- [NO]
## Have you added/updated Unit Tests and Integration Tests?
- [NO]Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/48WIP: redeploy gcp indexer with env var INDEXER_QUEUE_HOST2021-03-01T13:42:47ZOleksandr Kosse (EPAM)WIP: redeploy gcp indexer with env var INDEXER_QUEUE_HOSTredeploy gcp indexer with env var INDEXER_QUEUE_HOSTredeploy gcp indexer with env var INDEXER_QUEUE_HOSThttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/80WIP: feature/gonrg 16432021-03-01T13:45:24ZAliaksandr Lubouski (EPAM)WIP: feature/gonrg 1643indexer service testing branch with elastic 7 codeindexer service testing branch with elastic 7 codehttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/96Og ibm sync 2502212021-03-02T05:43:31ZMallikarjuna MallichettuOg ibm sync 250221Elastic search 7 and Fix on Unit test failsElastic search 7 and Fix on Unit test failsShrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/90Draft: Users/komakkar/exception handling fix2021-03-02T06:35:51ZKomal MakkarDraft: Users/komakkar/exception handling fixhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/102Draft: Updated os-core-common to 0.3.28 from 0.3.192021-03-02T08:10:11ZMANISH KUMARDraft: Updated os-core-common to 0.3.28 from 0.3.19MANISH KUMARMANISH KUMAR