Storage merge requestshttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests2022-09-15T23:29:08Zhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/437[BUG FIX] create error message using response from OPA2022-09-15T23:29:08ZRostislav Vatolinvatolinrp@gmail.com[BUG FIX] create error message using response from OPA## Type of change
- [X] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
Link to the issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/133
## Does this int...## Type of change
- [X] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
Link to the issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/133
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
No
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Tests are not passing when OPA flag is on
## What is the new/expected behavior?
Tests are passing when OPA flag is on
## Have you added/updated Unit Tests and Integration Tests?
Yes
## Any other useful information
NoM13 - Release 0.16Rostislav Vatolinvatolinrp@gmail.comRostislav Vatolinvatolinrp@gmail.comhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/438Cherry-pick 'Upgrade Core IBM Library for Release 0.15' into release/0.152022-06-20T20:38:38ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Upgrade Core IBM Library for Release 0.15' into release/0.15Original MR: !435Original MR: !435M12 - Release 0.15David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/440bulk API update2022-06-23T14:46:31ZAnkur Rawatbulk API updateBulk API update to incorporate MSI changesBulk API update to incorporate MSI changesAnkur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/442fix the nullpointer when acl is null2023-08-18T22:21:05ZYauheni Lesnikaufix the nullpointer when acl is nullWe are observing corrupted records with null acl on some environments. And trying to figure out how it could happen. But before we need to expose some fix for this bug, because it blocks reindex process for the entire index.We are observing corrupted records with null acl on some environments. And trying to figure out how it could happen. But before we need to expose some fix for this bug, because it blocks reindex process for the entire index.M13 - Release 0.16Yauheni LesnikauYauheni Lesnikauhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/444updated helm chart2022-06-29T13:59:49ZBhushan Radeupdated helm chartHelm chart updated by IBM teamHelm chart updated by IBM teamBhushan RadeBhushan Radehttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/445GONRG-5228: Added .dockerignore and .gitattributes2023-08-18T22:21:03ZKrisztian Molnar (EPAM)GONRG-5228: Added .dockerignore and .gitattributesM13 - Release 0.16Krisztian Molnar (EPAM)Krisztian Molnar (EPAM)https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/446added logs for verifying cache invalidation2022-07-07T12:33:37ZAnkur Rawatadded logs for verifying cache invalidationAdded logs to verify the invalidation of cache for legal tag deletion. Also added sleep timer for 2 seconds before making create calls.Added logs to verify the invalidation of cache for legal tag deletion. Also added sleep timer for 2 seconds before making create calls.Ankur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/448update Storage API spec to include Batch record read with normalization; fixe...2022-07-21T07:13:40ZVivek Ojhaupdate Storage API spec to include Batch record read with normalization; fixed...# Merge request template# Merge request templateVivek OjhaVivek Ojhahttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/450Remove Snapshot version from core-common- dependency2022-07-12T19:30:28ZShrikant GargRemove Snapshot version from core-common- dependency# Merge request template
https://community.opengroup.org/osdu/platform/system/storage/-/issues/135
Upgrading core-common as it is referring to SNAPSHOT version# Merge request template
https://community.opengroup.org/osdu/platform/system/storage/-/issues/135
Upgrading core-common as it is referring to SNAPSHOT versionM12 - Release 0.15Shrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/451Fix data inconsistency issue between metadata storage and blob storage2022-09-15T10:04:42ZAlok JoshiFix data inconsistency issue between metadata storage and blob storage## Issue
We've observed data inconsistency issues for some storage record objects where the latest version of record (available in metadata storage - i.e. cosmosDb) is soft deleted from blob storage (i.e. Azure Storage). This is causing ...## Issue
We've observed data inconsistency issues for some storage record objects where the latest version of record (available in metadata storage - i.e. cosmosDb) is soft deleted from blob storage (i.e. Azure Storage). This is causing exception for APIs like GET `/api/storage/v2/records/<id>` and POST `/api/storage/v2/query/records:batch`
## Solution
We propose to fix this data inconsistency by two changes:
- Clean up datastore as well, when cleaning up cloud storage, in case of commitBatch method exception in `PersistenceServiceImpl` (core logic change)
- For existing records having this issue: try restoring the data from blob storage. We are attempting to restore the latest record version (recorded in cosmosDb) from blob storage if its not foundM14 - Release 0.17Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/452Remove core common snapshot from root POM2022-07-13T15:35:25ZBrindaban DasRemove core common snapshot from root POMhttps://community.opengroup.org/osdu/platform/system/storage/-/issues/135
core-common-0.15.0-SNAPSHOT version is also referenced from Root POM. So, upgrading core-common version here also.https://community.opengroup.org/osdu/platform/system/storage/-/issues/135
core-common-0.15.0-SNAPSHOT version is also referenced from Root POM. So, upgrading core-common version here also.M12 - Release 0.15Brindaban DasBrindaban Dashttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/453update versions for core-lib-azure and os-core-common2022-09-16T08:25:03ZYauheni Lesnikauupdate versions for core-lib-azure and os-core-commonRemove snapshot versions for os-core-common and core-lib-azure for storage-core and storage-azureRemove snapshot versions for os-core-common and core-lib-azure for storage-core and storage-azureM13 - Release 0.16Yauheni LesnikauYauheni Lesnikauhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/454Upgraded log4j2 to fix vulnerabilities2022-07-14T04:39:50ZVineeth Guna [Microsoft]Upgraded log4j2 to fix vulnerabilities# Merge request template# Merge request templateVineeth Guna [Microsoft]Vineeth Guna [Microsoft]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/455upgrade version to '0.16.0-rc1'2022-08-03T07:35:07ZAshwani Pandeyupgrade version to '0.16.0-rc1'upgrade core lib IBM version to '0.16.0-rc1'upgrade core lib IBM version to '0.16.0-rc1'M13 - Release 0.16Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/456Update docs/tutorial/StorageService.md2023-08-18T22:21:02ZDebasis ChatterjeeUpdate docs/tutorial/StorageService.md# Merge request template# Merge request templateM13 - Release 0.16Debasis ChatterjeeDebasis Chatterjeehttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/457release MongoDB implementation2023-08-18T22:21:00ZAleh Shubko [EPAM]release MongoDB implementation# Merge request template# Merge request templateM13 - Release 0.16Aleh Shubko [EPAM]Aleh Shubko [EPAM]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/458Add deletion of a record after integration test2022-09-16T08:28:15ZYifei XuAdd deletion of a record after integration test## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud prov...## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
A record is not deleted after the integration test. It should be.
## What is the new/expected behavior?
All records should be deleted.
## Have you added/updated Unit Tests and Integration Tests?
Yes
## Any other useful informationM13 - Release 0.16Yifei XuOkoun-Ola Fabien HouetoYifei Xuhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/459Removed gcp reference module.2022-08-09T00:31:40ZRiabokon Stanislav(EPAM)[GCP]Removed gcp reference module.Removed GCP reference module.Removed GCP reference module.M13 - Release 0.16Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/462add extra validation for null acl on azure RecordMetadataRepository createOrU...2022-09-22T15:02:43ZYauheni Lesnikauadd extra validation for null acl on azure RecordMetadataRepository createOrUpdateWe are observing corrupted records with null acl on some environments. This changes should eliminate possibility of the issueWe are observing corrupted records with null acl on some environments. This changes should eliminate possibility of the issueM13 - Release 0.16Yauheni LesnikauYauheni Lesnikauhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/463Upgrade spring-webmvc.2022-09-16T02:20:00ZMorris EstepaUpgrade spring-webmvc.# Merge request template# Merge request templateM13 - Release 0.16Morris EstepaMorris Estepa