Storage merge requestshttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests2023-09-12T18:03:11Zhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/763Fixing vulnerabilities in Storage Service2023-09-12T18:03:11ZDaniel SchollFixing vulnerabilities in Storage ServiceThis change cleans up the POM file and fixes vulnerabilities in the service.This change cleans up the POM file and fixes vulnerabilities in the service.Daniel SchollDaniel Schollhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/721Fix instance name2023-06-29T16:52:15ZAleh Shubko [EPAM]Fix instance name# Merge request template# Merge request templateM19 - Release 0.22Madalyn MarabellaMadalyn Marabellahttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/719Fix instance name2023-06-29T15:34:33ZAleh Shubko [EPAM]Fix instance nameM19 - Release 0.22Madalyn MarabellaMadalyn Marabellahttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/264fix issue with impossibility of retrieving version with changed kind2023-08-18T22:22:59ZYauheni Lesnikaufix issue with impossibility of retrieving version with changed kindFix the issue when the version can't be retrieved because of changed kind.
Issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/89Fix the issue when the version can't be retrieved because of changed kind.
Issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/89M9 - Release 0.12https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/74Fix issue with persistable reference handling in converters;2023-08-18T22:27:34ZYauheni LesnikauFix issue with persistable reference handling in converters;This merge request is link with the another one in [os-core-common](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/23).
The issue was found in the **Storage GA** and concerned feature of the...This merge request is link with the another one in [os-core-common](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/23).
The issue was found in the **Storage GA** and concerned feature of the **Gson** lib, namely `getAsString()` method of the `com.google.gson.JsonElement` class with the subclasses (**JsonPrimitive**, **JsonObject**, etc). The thing is that the method overriden safely only for **JsomPrimitive** subclass, and if the field mapped as **JsonObject** one (persistableReference in the 'meta' part, in our case) this call throws **UnsupportedOperationException**.
My enhancement prevents undesirable throwing the exception. Also tests were extendedM1 - Release 0.1ethiraj krishnamanaiduFerris ArgyleDania Kodeih (Microsoft)Wladmir FrazaoJoeNitin-slbNeelesh ThakurKishore BattulaMatt WiseYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/742fix issue with records batch api as it drops valid record after conversion2023-08-07T19:40:18ZNeelesh Thakurfix issue with records batch api as it drops valid record after conversionRecord batch API incorrectly responding with `notFound` error for certain records combination, e.g.
```curl
curl --location 'http://host/api/storage/v2/query/records:batch' \
--header 'data-partition-id: opendes' \
--header 'frame-of-re...Record batch API incorrectly responding with `notFound` error for certain records combination, e.g.
```curl
curl --location 'http://host/api/storage/v2/query/records:batch' \
--header 'data-partition-id: opendes' \
--header 'frame-of-reference: units=SI;crs=wgs84;elevation=msl;azimuth=true north;dates=utc;' \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '{
"records": [
"opendes:master-data--Wellbore:ce85d8c5a4e75e2881e0f19a047e70b69191ea86dd13a71b9928cfa273f40e55",
"opendes:master-data--Wellbore:387fe57d5deac0533a78c7cf26f1d01324f442d506262913f17dad885c4a4b71"
]
}
```
Where `opendes:master-data--Wellbore:ce85d8c5a4e75e2881e0f19a047e70b69191ea86dd13a71b9928cfa273f40e55` has `AsIngestedCoordinates` as well valid `meta` block & `opendes:master-data--Wellbore:387fe57d5deac0533a78c7cf26f1d01324f442d506262913f17dad885c4a4b71` has `AsIngestedCoordinates` but no or empty `meta` block, then for such cases batch API responds with `notFound` for `opendes:master-data--Wellbore:387fe57d5deac0533a78c7cf26f1d01324f442d506262913f17dad885c4a4b71` record-id, even though there is nothing wrong with record or FoR conversion on record data.M20 - Release 0.23https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/568fix IT with retry2023-08-25T22:28:44ZNikhil Singh[MicroSoft]fix IT with retry# Merge request template# Merge request templateNikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/320Fix log4j2023-08-18T22:22:19ZArtem Dobrynin (EPAM)Fix log4jUpdated version to 2.17.0 to fix vulnerabilities
https://community.opengroup.org/groups/osdu/platform/-/epics/11
Part of the #103 seriesUpdated version to 2.17.0 to fix vulnerabilities
https://community.opengroup.org/groups/osdu/platform/-/epics/11
Part of the #103 seriesM10 - Release 0.13Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/543fix null pointer exception2022-12-27T05:33:19ZNikhil Singh[MicroSoft]fix null pointer exception**Introduction:**
Storage service is giving 500s due to null pointer exception when trying to convert ACL viewers into an array list but we dont have any such log for same. Also it is not reproducible because we never ingest a record wi...**Introduction:**
Storage service is giving 500s due to null pointer exception when trying to convert ACL viewers into an array list but we dont have any such log for same. Also it is not reproducible because we never ingest a record with acl fields empty. Hence we want to log the statement if ACL is empty and throw a NPE with proper reasoning.
**Exception:**
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)| at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)| at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)| at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)| at java.lang.Thread.run(Thread.java:750)|Caused by: java.lang.NullPointerException| at org.opengroup.osdu.storage.provider.azure.CloudStorageImpl.**hasViewerAccessToRecord(CloudStorageImpl.java:238)**| at org.opengroup.osdu.storage.provider.azure.CloudStorageImpl.read(CloudStorageImpl.java:284)| at org.opengroup.osdu.storage.provider.azure.CloudStorageImpl$$FastClassBySpringCGLIB$$47bbc639.invoke(<generated>)| at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)| at
ref #151Nikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/130fix null pointer exception when ancestry is empty2023-08-18T22:25:08ZSherman Yangfix null pointer exception when ancestry is emptyThis MR fixes a subtle null pointer exception which could occur in the follow line: !record.getAncestry().getParents().isEmpty()) when the ancestry has no parent. The getParents() would return null. The next call isEmpty() would cause a ...This MR fixes a subtle null pointer exception which could occur in the follow line: !record.getAncestry().getParents().isEmpty()) when the ancestry has no parent. The getParents() would return null. The next call isEmpty() would cause a null pointer exception when called on the returned null parents object.
The fix is to replace the line with the following line: !Collections.isEmpty(record.getAncestry().getParents())) Now if the getParents() method returns null, the Collections.isEmpty(null) call will correctly return true instead of null pointer exception as expected.
Corresponding unit tests have been updated accordingly.M5 - Release 0.8https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/426fix opa endpoint config (GONRG-5080)2022-09-10T09:09:15ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comfix opa endpoint config (GONRG-5080)## Type of change
- [x] Bug Fix
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does th...## Type of change
- [x] Bug Fix
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Property configuration of OPA endpoint combined with OPA service not working as it was planned,
value does not inject to the service, and during runtime, a NullPointer exception will occur when Storage tries to send request OPA endpoint.
And it no follow best practices as it said that configuration should be clean POJO objects.
https://www.baeldung.com/configuration-properties-in-spring-boot#simple-properties
## What is the new/expected behavior?
Property injects as it should be, via property file or via env variable.
## Have you added/updated Unit Tests and Integration Tests?
yes
## Any other useful information
This is a minor fix that will not require any changes in CSP configurations.M13 - Release 0.16Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/503Fix OqmSubscriberManager (legaltags_changed topic's subscriptions creation an...2022-09-13T16:38:45ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comFix OqmSubscriberManager (legaltags_changed topic's subscriptions creation and usage algorithm) (GONRG-4683)# Merge request template# Merge request templateM14 - Release 0.17Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/483Fix permissions for soft delete2022-09-27T21:50:27ZMadinabonu AlisherovaFix permissions for soft deleteFollowing endpoints only check for data viewer access currently:
* POST **/api/storage/v2/records/{id}:delete** (soft delete API)
* POST **/api/storage/v2/records/delete** (bulk delete API)
I updated soft delete API and now only users ...Following endpoints only check for data viewer access currently:
* POST **/api/storage/v2/records/{id}:delete** (soft delete API)
* POST **/api/storage/v2/records/delete** (bulk delete API)
I updated soft delete API and now only users with owner access can do the soft delete
issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/138M14 - Release 0.17Madinabonu AlisherovaMadinabonu Alisherovahttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/172Fix pod restarts on health check failues2021-08-30T21:34:36ZAlok JoshiFix pod restarts on health check failuesAfter pulling in this MR https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/149, we observed (in our internal ADO) that
- health check endpoint logs were missing from application insights and Log Analytics
- po...After pulling in this MR https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/149, we observed (in our internal ADO) that
- health check endpoint logs were missing from application insights and Log Analytics
- pods were not getting restarted when health checks were down
- [line51](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/devops/azure/chart/templates/deployment.yaml) line 51 says have to upgrade to Spring Boot 2.3 but I believe we are on 2.1.18.RELEASE
- Falling back to actuator health check by `spring-boot-starter-actuator`
@krveduruAlok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/40Fix: revert "Adding query all kinds bug fix, no longer filters by user"2023-08-18T22:29:36ZBrandt BealFix: revert "Adding query all kinds bug fix, no longer filters by user"• What is the change? Revert of previous commit
• What does it impact? AWS/Build
• How do I reproduce or test the change? N/A• What is the change? Revert of previous commit
• What does it impact? AWS/Build
• How do I reproduce or test the change? N/AM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/467Fix surefire2022-07-29T17:03:05ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comFix surefire# Merge request template# Merge request templatehttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/442fix the nullpointer when acl is null2023-08-18T22:21:05ZYauheni Lesnikaufix the nullpointer when acl is nullWe are observing corrupted records with null acl on some environments. And trying to figure out how it could happen. But before we need to expose some fix for this bug, because it blocks reindex process for the entire index.We are observing corrupted records with null acl on some environments. And trying to figure out how it could happen. But before we need to expose some fix for this bug, because it blocks reindex process for the entire index.M13 - Release 0.16Yauheni LesnikauYauheni Lesnikauhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/421fix token provisioning for anthos2022-05-25T15:51:00ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comfix token provisioning for anthos# Merge request template# Merge request templateM12 - Release 0.15Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/829Fix vulnerbilities2024-01-23T14:38:11ZBruce JinFix vulnerbilitiesFix existing critical CVEs and other Vulnerabilities:
https://community.opengroup.org/osdu/platform/system/storage/-/issues/196
https://community.opengroup.org/osdu/platform/system/storage/-/issues/194
https://community.opengroup.org/osd...Fix existing critical CVEs and other Vulnerabilities:
https://community.opengroup.org/osdu/platform/system/storage/-/issues/196
https://community.opengroup.org/osdu/platform/system/storage/-/issues/194
https://community.opengroup.org/osdu/platform/system/storage/-/issues/197
https://community.opengroup.org/osdu/platform/system/storage/-/issues/198
https://community.opengroup.org/osdu/platform/system/storage/-/issues/199
https://community.opengroup.org/osdu/platform/system/storage/-/issues/200
https://community.opengroup.org/osdu/platform/system/storage/-/issues/201
https://community.opengroup.org/osdu/platform/system/storage/-/issues/202
https://community.opengroup.org/osdu/platform/system/storage/-/issues/203M23 - Release 0.26Bruce JinBruce Jinhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/306fix whitesource issues2023-08-18T22:22:29ZYauheni Lesnikaufix whitesource issuesIssue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/108Issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/108M11 - Release 0.14Yauheni LesnikauYauheni Lesnikau