Storage merge requestshttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests2023-05-10T17:48:30Zhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/530Whitesource update vulnerable dependencies2023-05-10T17:48:30ZMykyta SavchukWhitesource update vulnerable dependenciesUpdated common libraries, but because spring boot was updated to 2.7.2 there, so I added some code changes:
- Migrated from springfox to springdoc, because springfox is not compatible with this version of spring boot
- The new version of...Updated common libraries, but because spring boot was updated to 2.7.2 there, so I added some code changes:
- Migrated from springfox to springdoc, because springfox is not compatible with this version of spring boot
- The new version of spring boot found dependency cycle, so I removed it with @LazyM15 - Release 0.18Mykyta SavchukMykyta Savchukhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/531Correct the attribute name to "PreloadFilePath" for os-storage2023-08-18T22:20:52ZXiangliang MengCorrect the attribute name to "PreloadFilePath" for os-storage# Merge request template# Merge request templateM15 - Release 0.18Okoun-Ola Fabien HouetoXiangliang MengOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/532consume latest version of core common library and update the way of getting p...2022-12-12T22:51:21ZKelly Zhouconsume latest version of core common library and update the way of getting parent recordsPlease refer to issue [here](https://community.opengroup.org/osdu/platform/system/storage/-/issues/145)Please refer to issue [here](https://community.opengroup.org/osdu/platform/system/storage/-/issues/145)M15 - Release 0.18https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/533Upgrade spring-beans2023-08-18T22:20:50ZXiangliang MengUpgrade spring-beans# Merge request template# Merge request templateM15 - Release 0.18Okoun-Ola Fabien HouetoXiangliang MengOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/534Delete all versions of the blob when purging a record2022-12-09T10:53:01ZAlok JoshiDelete all versions of the blob when purging a recordPlease refer to [issue ](https://community.opengroup.org/osdu/platform/system/storage/-/issues/147)
This change solves the samePlease refer to [issue ](https://community.opengroup.org/osdu/platform/system/storage/-/issues/147)
This change solves the sameM15 - Release 0.18Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/535Storage Performance (GONRG-4754)2022-12-13T00:35:58ZRiabokon Stanislav(EPAM)[GCP]Storage Performance (GONRG-4754)## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] I...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the new/expected behavior?
Fixed NPE. Key is Null.
```
{"code":500,"reason":"Server error.","message":"An unknown error has occurred."}
{"code":500,"reason":"Error writing record.","message":"The server could not process your request at the moment."}
```
https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/osm/-/merge_requests/31/M15 - Release 0.18Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/538GONRG-5890-Change-to-GC2023-08-18T22:20:48ZEldor Pulatov [EPAM/GCP]GONRG-5890-Change-to-GC# Merge request template# Merge request templateM15 - Release 0.18Eldor Pulatov [EPAM/GCP]Eldor Pulatov [EPAM/GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/539ADR 92: Populate recordBlocks in record modification events2022-12-08T22:56:14ZAbhishek NandaADR 92: Populate recordBlocks in record modification events## Type of change
- [x] Bug Fix
- [X] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
[ADR 92](https://community.opengroup.org/osdu/platform/system/home/-/issues/92)
## Does this introduce a chang...## Type of change
- [x] Bug Fix
- [X] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
[ADR 92](https://community.opengroup.org/osdu/platform/system/home/-/issues/92)
## Does this introduce a change in the core logic?
- [YES/NO] No
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [x] IBM (Added an additional field (hash) to the document db. Auto populated for all other providers.)
## Does this introduce a breaking change?
- [YES/NO] No
## What is the current behavior?
- skipdupes only works for changes in RecordData and ignores changes in RecordMetadata.
- Record modification events do not contain info on which section was updated
## What is the new/expected behavior?
- skipdupes=true now handles for change in any of RecordData and RecordMetadata fields
- This also implements [ADR 92](https://community.opengroup.org/osdu/platform/system/home/-/issues/92) which populates a recordBlocks field with the fields updated
- skipdupes now uses the recordBlock field to check if any updates are made to the record
## Have you added/updated Unit Tests and Integration Tests?
Unit Tests for all new files and new lines with 100% coverage has been added
## Any other useful information
The hashes of data and meta field are now saved in metadata which is used to validate if any changes were made and prevents and extra call to the blob storage for each record.
This is used both in skipdupes=true scenario as well as to populate the recordBlocks field.
This brings around a 50% performance improvement in skipdupes=true scenariosM15 - Release 0.18https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/540added ability to config redis password and ssl (GONRG-5301)2022-12-13T00:28:53ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comadded ability to config redis password and ssl (GONRG-5301)added ability to config redis password and ssladded ability to config redis password and sslM15 - Release 0.18Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/542Changed CRC32C to MD5 to calculate a checksum (GONRG-5884)2022-12-12T23:54:41ZRiabokon Stanislav(EPAM)[GCP]Changed CRC32C to MD5 to calculate a checksum (GONRG-5884)## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IB...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the new/expected behavior?
Upgraded obm lib. look at https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp/-/merge_requests/57.
Changed an algorithm to generate check sum.M15 - Release 0.18Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/544upgrading vulnerable dependencies2022-11-22T11:54:45ZAshwani Pandeyupgrading vulnerable dependenciesUpgraded dependencies to fix below vulnerabilities:
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21875
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21...Upgraded dependencies to fix below vulnerabilities:
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21875
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21868
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21867
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18898
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18896
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18894
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18892
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18886
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18885
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18883
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18880
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18879
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18875
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18874
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24810
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24803
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24798
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24797
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24795
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24794
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24253
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24251
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24250
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24249
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24246
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21874
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21860
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21857
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21851
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21294
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21293
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20014
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20013
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20011
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20006
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/19440
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/19436
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/16483
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/16482
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/12481M15 - Release 0.18Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/545Squashed commit of the following2022-11-21T16:40:42ZMorris EstepaSquashed commit of the followingupdate buildspec & pom versionsupdate buildspec & pom versionsM15 - Release 0.18Morris EstepaMorris Estepahttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/546Update Storage to be Collaboration Context Aware2022-12-09T15:57:24ZMina OtgonboldUpdate Storage to be Collaboration Context Aware# Merge request template
This MR updates all APIs in storage to be collaboration context-aware. This functionality is behind a collaboration feature flag which is set to false by default. The functionality of the existing storage service...# Merge request template
This MR updates all APIs in storage to be collaboration context-aware. This functionality is behind a collaboration feature flag which is set to false by default. The functionality of the existing storage service will not be changed with this feature flag set to false.
When it is set to true the old functionality is still not changed however you can work with Records in new contexts using the x-collaboration header when it is optionally provided.
[This core-common MR](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/183) has to be merged prior to this PR.
This aligns with the ADR below. This MR does not include the new Record changed topic. The current record changed topic is not triggered by any changes when a collaboration context is provided.
References
- [ADR](https://community.opengroup.org/osdu/platform/system/storage/-/issues/149)M15 - Release 0.18https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/550Removing the azure message bus message id2022-12-14T05:51:04ZThiago SenadorRemoving the azure message bus message id## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IB...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Azure message bus message id is composed by correlation id. We have applications that use the same correlation in multiple requests to storage and that is creating duplicated messages with incorrect behavior.
## What is the new/expected behavior?
Azure message bus message id is going to be assigned by Azure itself, guaranteeing its uniqueness.
## Have you added/updated Unit Tests and Integration Tests?
No
## Any other useful informationM15 - Release 0.18https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/551GONRG-6036: Add vars for opa2023-08-18T22:20:47ZYauheni Rykhter (EPAM)GONRG-6036: Add vars for opa# Merge request template# Merge request templateM15 - Release 0.18Yauheni Rykhter (EPAM)Yauheni Rykhter (EPAM)https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/552upgrading vulnerability dependency2022-11-24T05:29:03ZAshwani Pandeyupgrading vulnerability dependencyUpgraded dependencies to fix below vulnerabilities:
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21875
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21...Upgraded dependencies to fix below vulnerabilities:
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21875
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21868
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21867
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18898
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18896
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18894
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18892
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18886
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18885
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18883
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18880
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18879
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18875
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/18874
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24810
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24803
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24798
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24797
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24795
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24794
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24253
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24251
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24250
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24249
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/24246
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21874
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21860
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21857
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21851
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21294
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/21293
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20014
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20013
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20011
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/20006
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/19440
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/19436
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/16483
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/16482
https://community.opengroup.org/osdu/platform/system/storage/-/security/vulnerabilities/12481M15 - Release 0.18Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/553Include collaboration context when pushing to new record changed topic2024-03-19T02:18:18ZMina OtgonboldInclude collaboration context when pushing to new record changed topic## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IB...## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
If collaboration context is provided in the request: Azure message bus neither sends a message to recordstopic (i.e. let's call it an old topic) nor recordsevent (i.e. let's call it a new topic)
If collaboration context is not provided in the request: Azure message bus sends messages to the old topic (recordstopic) only
## What is the new/expected behavior?
If collaboration context is provided in the request: Azure message bus sends message to a new topic (recordsevent) that contains x-collaboration directives in message properties.
If collaboration context is not provided in the request: Azure message bus sends message to new topic (recordsevent) and old topic (recordstopic)
## Have you added/updated Unit Tests and Integration Tests?
Yes
## Any other useful information
[ADR](https://community.opengroup.org/osdu/platform/system/storage/-/issues/149)
This [core-lib-azure MR](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/257) has to merge to master before this MR.M15 - Release 0.18https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/554Revert "Merge branch 'gcp-password-for-redis' into 'master'"2023-08-18T22:20:45ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRevert "Merge branch 'gcp-password-for-redis' into 'master'"# Merge request template# Merge request templateM15 - Release 0.18Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/556added changes to fix the azure_code_coverage job2022-12-01T09:24:14Zshivani karipeadded changes to fix the azure_code_coverage job- This change will fix the azure code coverage job and produce the code coverage results
- Jacoco plugin is used for the code coverage
- issue ref: https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-prov...- This change will fix the azure code coverage job and produce the code coverage results
- Jacoco plugin is used for the code coverage
- issue ref: https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/244M15 - Release 0.18shivani karipeshivani karipehttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/560added spring-boot-maven plugin version2022-11-30T06:40:51ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comadded spring-boot-maven plugin version## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provi...## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [x] Azure
- [x] Google Cloud
- [x] IBM
- [x] Common code
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
Projects that do not specify the spring-boot-maven plugin version will not compile, since the latest version of the spring-boot-maven plugin suppose to build spring-boot V3 projects with Java 17M15 - Release 0.18Chad LeongChad Leong