Indexer merge requestshttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests2023-05-23T10:29:22Zhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/551remove reprocessing in case storage respond with no valid records2023-05-23T10:29:22ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comremove reprocessing in case storage respond with no valid recordsM19 - Release 0.22Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/180remove reference of old maven slb-des-ext-collaboration2023-08-18T15:07:34ZNeelesh Thakurremove reference of old maven slb-des-ext-collaborationRemove references of old maven ADO package manager from `slb-des-ext-collaboration`. These configurations are not longer in use.Remove references of old maven ADO package manager from `slb-des-ext-collaboration`. These configurations are not longer in use.M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/265Remove Page Size check while querying Storage API2022-07-12T15:41:14ZVibhuti Sharma [Microsoft]Remove Page Size check while querying Storage APIVibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/198Remove logic of act as for indexer-queue(GONRG-3013)2021-09-15T08:50:23ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRemove logic of act as for indexer-queue(GONRG-3013)# Description:
Unhardcode cloud task service account
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
# Changes in:
- [x] GCP
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable...# Description:
Unhardcode cloud task service account
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
# Changes in:
- [x] GCP
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [ ] Existing Tests pass
- [ ] Verified functionality locally
- [ ] Self Reviewed my code for formatting and complex business logic.M9 - Release 0.12Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/514remove legacy code that's not being used by any API2023-04-24T14:30:50ZNeelesh Thakurremove legacy code that's not being used by any APIM18 - Release 0.21https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/534remove infra provisioning from java code(GONRG-7073)2023-05-12T10:38:34ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comremove infra provisioning from java code(GONRG-7073)M18 - Release 0.21Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/361Remove indexer-reference from .fossa-yml modules2022-09-25T13:24:38ZDavid Diederichd.diederich@opengroup.orgRemove indexer-reference from .fossa-yml modulesThe reference module was removed in !353, but the fossa configuration was missed.
As a result, the 'fossa-analyze' steps were failing, unable to list dependencies in the non-existent project.The reference module was removed in !353, but the fossa configuration was missed.
As a result, the 'fossa-analyze' steps were failing, unable to list dependencies in the non-existent project.M13 - Release 0.16David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/558Remove feature flag for geoshape decimation2023-07-05T18:53:03ZZhibin MaiRemove feature flag for geoshape decimationThis MR is to remove the feature flag for geoshape decimation as geoshape decimation has been deployed since M16 and the feedback is positive:
1. It boosts the GIS map performance 20-30 times (in terms of search and data fetch) after the...This MR is to remove the feature flag for geoshape decimation as geoshape decimation has been deployed since M16 and the feedback is positive:
1. It boosts the GIS map performance 20-30 times (in terms of search and data fetch) after the geoshape is decimated in the client deployments.
2. It speeds up the indexing performance as the overhead on geoshape decimation is offset by the performance improvement on the ElasticSearch indexing (geoshape indexing)M19 - Release 0.22Zhibin MaiZhibin Maihttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/50Removed unhardcode repositories and distribution management from all poms (GO...2023-08-18T15:17:35ZRiabokon Stanislav(EPAM)[GCP]Removed unhardcode repositories and distribution management from all poms (GONRG-1056)## Type of change
- [ ] Bug Fix
- [X] Feature
ADR: https://community.opengroup.org/osdu/platform/system/home/-/issues/55
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider...## Type of change
- [ ] Bug Fix
- [X] Feature
ADR: https://community.opengroup.org/osdu/platform/system/home/-/issues/55
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] AWS
- [X] Azure
- [X] GCP
- [X] IBM
## Does this introduce a breaking change?
- [YES]
## What is the new/expected behavior?
This MR brings changes to POMs (root, azure, gcp) to improve (unhardcode) the way how repositories are being pointed. It now uses set of properties defined in the `.mvn/community-maven.settings.xml`
## Have you added/updated Unit Tests and Integration Tests?
- [YES]
## Any other useful informationM1 - Release 0.1Dmitriy RudkoRostislav Dublin (EPAM)Dmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/384Removed retry attempts and call to storage service2022-10-28T11:24:00ZHarshika DhootRemoved retry attempts and call to storage serviceM15 - Release 0.18Harshika DhootHarshika Dhoothttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/353Removed gcp reference module (GONRG-4669)2022-08-09T00:46:45ZRiabokon Stanislav(EPAM)[GCP]Removed gcp reference module (GONRG-4669)Removed gcp reference module.
https://community.opengroup.org/osdu/platform/system/indexer-service/-/security/vulnerabilities/7077Removed gcp reference module.
https://community.opengroup.org/osdu/platform/system/indexer-service/-/security/vulnerabilities/7077M13 - Release 0.16Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/98Removed elastic 6.6.2 from int tests ibm2021-03-01T12:49:27ZRiabokon Stanislav(EPAM)[GCP]Removed elastic 6.6.2 from int tests ibmRemoved elastic 6.6.2 from int tests ibm
## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?...Removed elastic 6.6.2 from int tests ibm
## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [X] IBM
## Does this introduce a breaking change?
- [NO]
## Have you added/updated Unit Tests and Integration Tests?
- [NO]Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/460Removed audience property (GONRG-6243)2023-01-12T23:52:48ZYurii Ruban [EPAM / GCP]Removed audience property (GONRG-6243)# Description:
Removed audience property
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
# Changes in:
- [x] GCP# Description:
Removed audience property
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
# Changes in:
- [x] GCPM16 - Release 0.19Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/610Remediate guava, netty-handler vulnerabilities in Azure2023-09-08T07:01:40ZThulasi Dass SubramanianRemediate guava, netty-handler vulnerabilities in Azure# Change details
* upgrade `com.google.guava:guava` to latest version `32.1.2-jre`
* upgrade `io.netty:netty-bom` version to `4.1.96.Final`
# Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBM# Change details
* upgrade `com.google.guava:guava` to latest version `32.1.2-jre`
* upgrade `io.netty:netty-bom` version to `4.1.96.Final`
# Changes in:
* [ ] GCP
* [x] Azure
* [ ] AWS
* [ ] IBMM21 - Release 0.24Thulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/125Release 0.7 hotfix #16 - array-of-objects' support implementation2021-03-16T20:15:20ZDmitriy RudkoRelease 0.7 hotfix #16 - array-of-objects' support implementation#16 - Fix indexing ' array of object' for all CSPs
See merge request osdu/platform/system/indexer-service!123
(cherry picked from commit feb78aa0d2287cd99405390c62a40592bd94053e)
2a533b3f #16 - Fix indexing ' array of object' for all ...#16 - Fix indexing ' array of object' for all CSPs
See merge request osdu/platform/system/indexer-service!123
(cherry picked from commit feb78aa0d2287cd99405390c62a40592bd94053e)
2a533b3f #16 - Fix indexing ' array of object' for all CSPs
e0e84f13 #16 - Fix unit tests for SchemaToStorageFormatImplTest
5824b1ae Merge branch 'master' into hotfix/#16-array-of-objects
ae7f09d5 resolve unwanted cloud tasks retry by fixing exception handlerDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/143relax validation for internal schema fragments2023-08-18T15:08:13ZNeelesh Thakurrelax validation for internal schema fragmentsProblem: indexer schema (schema service) parsing validation has broken existing schema consumption, resulting in record not being indexed
Explanation: Kind schema can be composed of fragments. These fragments can be internal or external...Problem: indexer schema (schema service) parsing validation has broken existing schema consumption, resulting in record not being indexed
Explanation: Kind schema can be composed of fragments. These fragments can be internal or external to a schema. Internal schema fragments are required to be defined on the schema ('definitions' block) whereas external fragments (e.g. AbstractFeatureCollection) can be registered separately with schema service. Schema service resolves these reference when a user makes request to kind schema.
External fragments follows fully qualified kind names: <authority>:<source>:<individualType>:<major>.<minor>.<patch>, e.g. AbstractFeatureCollection will be registered: opendes:wks:AbstractFeatureCollection:1.0.0, whereas internal fragments do not have same requirements.
Recent indexer updates expects all references to have fully qualified names (internal or external). This MR relax the the validation thus enabling internal schema fragments parsing by indexerM5 - Release 0.8https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/336reindex status code fix2023-02-04T05:35:46ZSHEFFALI JAINreindex status code fixCurrently re-index API sends out 200 response code no matter reindexing activity is performed or not.
Now, sending out 500 status code if any of the kind is not reindexed.
If successful in pitting messages in servcie bus then 200.
Also r...Currently re-index API sends out 200 response code no matter reindexing activity is performed or not.
Now, sending out 500 status code if any of the kind is not reindexed.
If successful in pitting messages in servcie bus then 200.
Also reindex by kind will show the same behaviour.
Advancement : Saving all the response for each kind in hashmap. That can be used to send as response to client for better tracking of what kind is not reindexed.SHEFFALI JAINSabarish K R ESHEFFALI JAINhttps://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/262Reindex API Improvement2022-01-21T10:22:44ZVibhuti Sharma [Microsoft]Reindex API ImprovementAddresses https://community.opengroup.org/osdu/platform/system/storage/-/issues/106Addresses https://community.opengroup.org/osdu/platform/system/storage/-/issues/106Vibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/209refresh auth token on long running reindex request2023-08-18T15:07:01ZNeelesh Thakurrefresh auth token on long running reindex requestRefresh auth token on long running reindex request otherwise reindex will terminate after 60mins.Refresh auth token on long running reindex request otherwise reindex will terminate after 60mins.M9 - Release 0.12https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/615Refactoring send method2023-10-30T04:17:33ZHarshika DhootRefactoring send methodAdding logs such that for any 4xx or 5xx calls we are able to trace the flow using correlation id, urls, http methods, which will enable us to perform the debugging faster and also at the places where we are blocked because of no correla...Adding logs such that for any 4xx or 5xx calls we are able to trace the flow using correlation id, urls, http methods, which will enable us to perform the debugging faster and also at the places where we are blocked because of no correlation id
ADME instance compatibility:![indexeradme](/uploads/cfc50131a39cc3e6c93a1198eb5330d8/indexeradme.png)
Test case scenarios
1. green case: ![indexgreen](/uploads/6e8e751b027239ab784f97559cb494e9/indexgreen.png)
2. 4xx case: ![indexe4xx](/uploads/63ab582f32eadb8a4384469aeaeb1ee0/indexe4xx.png)
3. 5xx case: ![indexe5xx](/uploads/6143b936a915eee3397aeb16bcba33c0/indexe5xx.png)Harshika DhootHarshika Dhoot