OS Core Lib Azure merge requestshttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests2024-03-20T11:36:24Zhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/340Added oid validation2024-03-20T11:36:24ZDeepa KumariAdded oid validationOID validation addition to address the issue: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/166OID validation addition to address the issue: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/166M23 - Release 0.26Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/339[MSOSDU-36231]Added OID Validation utilities and tests2024-03-12T11:59:35ZDeepa Kumari[MSOSDU-36231]Added OID Validation utilities and testsAdded interaction with Graph API to validate OID. Related issue: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/166Added interaction with Graph API to validate OID. Related issue: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/166M23 - Release 0.26Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/337update documentation in readme2024-03-19T17:14:21ZVidyaDharani Lokamupdate documentation in readme- Added the Jacoco code coverage report in the README.
- Updated the prerequisites with links.- Added the Jacoco code coverage report in the README.
- Updated the prerequisites with links.M23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/335Java 17 migration2024-03-21T05:54:33ZVidyaDharani LokamJava 17 migration# Description:
Migration from Java 8 to Java 17.
# How to test:
Via Unit and Integration tests.
# Changes include:
* [x] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:...# Description:
Migration from Java 8 to Java 17.
# How to test:
Via Unit and Integration tests.
# Changes include:
* [x] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
* [x] Common code
# Dev Checklist:
* [ ] Added Unit Tests, wherever applicable.
* [x] Updated the Readme, if applicable.
* [x] Existing Tests pass
* [x] Verified functionality locally
* [ ] Self Reviewed my code for formatting and complex business logic.
# Other comments:
* Upgraded 'lombok' & 'jacoco' plugin version to generate code-coverage report
* Changed TestRunner 'PowerMockRunner' to 'MockitoJUnitRunner'
* Removed 'powermock-api-mockito2' & 'powermock-module-junit4' dependencies
* Updated existing testcases to use junit-5M23 - Release 0.26VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/332429 exception handling for bulk update operation in storage2024-02-15T07:02:26ZAnkur Rawat429 exception handling for bulk update operation in storageCurrently, bulk insertion in cosmos DB functionality returns 500 in case of 429 error form cosmos DB. Updated the functionality to send 429 received from Cosmos DBCurrently, bulk insertion in cosmos DB functionality returns 500 in case of 429 error form cosmos DB. Updated the functionality to send 429 received from Cosmos DBAnkur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/328CosmosDB 429 Exception handling in Azure2024-03-19T17:14:22ZAnkur RawatCosmosDB 429 Exception handling in AzureCurrently, bulk insertion in cosmos DB functionality returns 500 in case of 429 error form cosmos DB.
Updated the functionality to send 429 received from Cosmos DB.Currently, bulk insertion in cosmos DB functionality returns 500 in case of 429 error form cosmos DB.
Updated the functionality to send 429 received from Cosmos DB.M23 - Release 0.26Ankur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/325make tenant cache thread-safe2024-01-17T05:09:57ZAlok Joshimake tenant cache thread-safeIn scenarios where there are many partitions (more than 50), we are seeing intermittent null values when accessing partition info from cache. Making the tenant cache thread safe would prevent this.In scenarios where there are many partitions (more than 50), we are seeing intermittent null values when accessing partition info from cache. Making the tenant cache thread safe would prevent this.M23 - Release 0.26Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/324update release version2024-01-10T17:29:10ZChristophe Monginupdate release version## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have...## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have added tests to cover my changes.
* [YES/NO/NA] All new and existing tests passed.
* [YES/NO/NA] My code follows the code style of this project.
* [YES/NO/NA] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [YES/NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/323Fix s360 vulnerability (json version)2024-01-09T22:29:09ZChristophe MonginFix s360 vulnerability (json version)## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have...## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have added tests to cover my changes.
* [YES/NO/NA] All new and existing tests passed.
* [YES/NO/NA] My code follows the code style of this project.
* [YES/NO/NA] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [YES/NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/321Fix support more than 10 patch operations in a single request2024-01-22T17:21:18ZYurii KondakovFix support more than 10 patch operations in a single request## Type of change
- [x] Bug Fix
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
Currently,...## Type of change
- [x] Bug Fix
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] Google Cloud
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
Currently, we have a limit of 10 operations on the maximum number of patch operations due to limitations on the part of Azure Cosmos DB.
[Partial document update in Azure Cosmos DB](https://learn.microsoft.com/en-us/azure/cosmos-db/partial-document-update)
For the API consumers, this number is less by 2, i.e. 8, as we add two more patch operations internally (at storage service).
> It's possible to execute multiple patch operations on a single document. The maximum limit is 10 operations. Multi-document patch: Multiple documents within the same partition key can be patched as a part of a transactionM23 - Release 0.26Yurii KondakovYurii Kondakovhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/317Cherry-pick 'Full Upgrade of First Party Library Dependencies' into release/0.252023-12-12T19:21:55ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Full Upgrade of First Party Library Dependencies' into release/0.25**Original MR**: !316
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !316
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/pipelines/new?ref=cherry-pick-for-316)M22 - Release 0.25David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/316Full Upgrade of First Party Library Dependencies2023-12-12T18:02:47ZDavid Diederichd.diederich@opengroup.orgFull Upgrade of First Party Library DependenciesThis generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep all dependent libraries up to date.
This upgrade can be merged immediately without further approval if the C...This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep all dependent libraries up to date.
This upgrade can be merged immediately without further approval if the CI pipeline reports success.
If this MR has failed, we need to work with the maintainers and affected provider teams to find a solution.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: ce77bafdba9957084a6617509ac49fb807d72c8d
Maven: 0.26.0-SNAPSHOT
```
| Maven Dependencies | _Root_ |
| ------------------ | ---------- |
| os-core-common | 0.25.0-rc3 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: b63f8b0bd6c71c5fb820821e120e418492d3319d
Maven: 0.26.0-SNAPSHOT
```
| Maven Dependencies | _Root_ |
| ------------------ | ------ |
| os-core-common | 0.25.0 |M22 - Release 0.25https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/315[MSCOSDU-1837] fix: Upgrade json version to remediate vulnerability2023-12-07T13:20:14ZThulasi Dass Subramanian[MSCOSDU-1837] fix: Upgrade json version to remediate vulnerability# Details
- upgraded `org.json:json` to latest version `20231013` to remediate vulnerability
- Reference : https://mvnrepository.com/artifact/org.json/json/20231013# Details
- upgraded `org.json:json` to latest version `20231013` to remediate vulnerability
- Reference : https://mvnrepository.com/artifact/org.json/json/20231013M22 - Release 0.25Thulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/314Upgrade sb library to latest version2023-11-29T16:03:36ZAlok JoshiUpgrade sb library to latest versionUse latest version of [azure-servicebus](https://mvnrepository.com/artifact/com.microsoft.azure/azure-servicebus). We are seeing an **intermittent** issue with delete subscription workflow where the client connection for the deleted subs...Use latest version of [azure-servicebus](https://mvnrepository.com/artifact/com.microsoft.azure/azure-servicebus). We are seeing an **intermittent** issue with delete subscription workflow where the client connection for the deleted subscription is not being properly closed. The connection close action happens [here ](https://community.opengroup.org/osdu/platform/system/notification/-/blob/master/provider/notification-azure/src/main/java/org/opengroup/osdu/notification/provider/azure/messageBus/SubscriptionManagerImpl.java?ref_type=heads#L129) for Notification service. The issue we see that the client connection is still open (thus, message pump still being attempted; a lot of exception entries from [this](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/src/main/java/org/opengroup/osdu/azure/servicebus/AbstractMessageHandler.java?ref_type=heads#L83)) even after close is "successful" ([logger info](https://community.opengroup.org/osdu/platform/system/notification/-/blob/master/provider/notification-azure/src/main/java/org/opengroup/osdu/notification/provider/azure/messageBus/SubscriptionManagerImpl.java?ref_type=heads#L130) can be seen in Appinsights logs). This is causing an explosion in exception logs, making it expensive. This upgrade hopefully fixes the issue with client connection closing.
We are also working on creating a support ticket with MSFT about the issue and another Gitlab issue to [migrate](https://aka.ms/azsdk/java/migrate/sb) to the suggested SB library.M22 - Release 0.25Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/313Upgrade First Party Library Dependencies for Release 0.242023-10-13T17:57:09ZChad LeongUpgrade First Party Library Dependencies for Release 0.24This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 22b373a3bec786c96cedfa9a39bae0de487c289b
Maven: 0.24.0-SNAPSHOT
```
```
No Maven dependencies to show. (No first-party dependencies, and all third-party dependencies are hidden)
```
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: 506d58b8d9f7283b02fbf8e2d38bbbca2fa869f2
Maven: 0.24.0-SNAPSHOT
```
```
No Maven dependencies to show. (No first-party dependencies, and all third-party dependencies are hidden)
```M21 - Release 0.24https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/311Cleanup documentdb dependency and code references2023-10-05T12:19:20ZThulasi Dass SubramanianCleanup documentdb dependency and code references## Changes
* exclude unused `documentdb-bulkexecutor` dependency
* cleanup the associated code w.r.t `documentdb-bulkexecutor`
# Dev Checklist:
* [x] Existing Tests pass
* [x] Verified functionality locally
* [x] Self Reviewed code for...## Changes
* exclude unused `documentdb-bulkexecutor` dependency
* cleanup the associated code w.r.t `documentdb-bulkexecutor`
# Dev Checklist:
* [x] Existing Tests pass
* [x] Verified functionality locally
* [x] Self Reviewed code for references/call usages of `documentdb-bulkexecutor` before cleanup. Only `Storage service` references found, yet it was dead-code methods & no usages inside any API-services call flow.
* [x] used inspect unused code usages in IntelliJ IDEM21 - Release 0.24Thulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/310Upgrading core common version to utilize the latest redis fix2023-09-19T19:10:40ZDavid Diederichd.diederich@opengroup.orgUpgrading core common version to utilize the latest redis fixThis ~"Direct patch" updates the core common version for the %"M20 - Release 0.23" release. It was not brought in as a ~"Cherry-pick" because the default branch is already at `0.24.0-rc3`, and I didn't want to downgrade it.This ~"Direct patch" updates the core common version for the %"M20 - Release 0.23" release. It was not brought in as a ~"Cherry-pick" because the default branch is already at `0.24.0-rc3`, and I didn't want to downgrade it.M20 - Release 0.23David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/307Update os-core-common version to '0.24.0-rc3'2023-10-13T17:59:55ZThulasi Dass SubramanianUpdate os-core-common version to '0.24.0-rc3'## Change Details
* update `os-core-common` version to `0.24.0-rc3`
* Associated [os-core-common MR 232](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/232)## Change Details
* update `os-core-common` version to `0.24.0-rc3`
* Associated [os-core-common MR 232](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/232)M21 - Release 0.24Thulasi Dass SubramanianThulasi Dass Subramanianhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/304Cherry-pick 'Remediate guava vulnerability' into release/0.232023-10-13T17:59:56ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Remediate guava vulnerability' into release/0.23**Original MR**: !303
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !303
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/pipelines/new?ref=cherry-pick-for-303)M20 - Release 0.23David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/303Remediate guava vulnerability2023-10-13T17:59:55ZThulasi Dass SubramanianRemediate guava vulnerability* upgrade `com.google.guava:guava` to latest version 32.1.2-jre* upgrade `com.google.guava:guava` to latest version 32.1.2-jreM20 - Release 0.23Thulasi Dass SubramanianThulasi Dass Subramanian