OS Core Lib Azure merge requestshttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests2021-06-21T12:28:14Zhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/116Extending cosmosStore class to support CRUD on system cosmos db2021-06-21T12:28:14ZAman VermaExtending cosmosStore class to support CRUD on system cosmos dbProblem:
====
The `cosmosStore` class today is the facade to perform CRUD on data partition related cosmos dbs. All the methods in this class are tightly coupled with data-partition-id. Since the system data resources won't be coupled wi...Problem:
====
The `cosmosStore` class today is the facade to perform CRUD on data partition related cosmos dbs. All the methods in this class are tightly coupled with data-partition-id. Since the system data resources won't be coupled with a data-partition-id, the existing facade can't support CRUD on system resources.
Changes in MR:
===
1. Enhanced the current CosmosClientFactory to return system level cosmos clients too.
2. Overloaded existing methods. The new methods don't accept data-partition-id parameter
3. Minor code refactoring
Abandoning this MR. Find new MR here: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/118Aman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/120[WIP] Added support for File Share in core lib azure2021-06-24T06:17:49ZAalekh Jain[WIP] Added support for File Share in core lib azureThis MR targets to add the implementation for File Share in core lib azure.
cc: @vineethguna , @kibattulThis MR targets to add the implementation for File Share in core lib azure.
cc: @vineethguna , @kibattulhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/115Draft: Gateway mode enabled2021-06-26T08:27:52ZAbhiram BondadaDraft: Gateway mode enabledGateway mode enabled in "createCosmosClient" method of CosmosClientFactoryImplGateway mode enabled in "createCosmosClient" method of CosmosClientFactoryImplAbhiram BondadaAbhiram Bondadahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/122Draft: Added Cache parameter2021-06-26T10:54:12ZAbhiram BondadaDraft: Added Cache parametermade changes to enable gateway mode, with changes in PartitionInfoAzure using additional getters for gateway endpointmade changes to enable gateway mode, with changes in PartitionInfoAzure using additional getters for gateway endpointhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/117Draft: Added Initial configuration for circuitbreakers2021-06-29T05:25:26ZRonak SakhujaDraft: Added Initial configuration for circuitbreakershttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/135Draft: Making changes to test cb2021-08-11T19:50:24ZRonak SakhujaDraft: Making changes to test cbhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/70Set App Insights user_Id value for requests and exceptions2021-08-24T11:57:28ZKelly DomicoSet App Insights user_Id value for requests and exceptionsSet App Insights "user_Id" value for requests and exceptions. This value is copied from the customDimensions object. Setting the user_Id makes it easier for creating dashboards on the Azure portal.Set App Insights "user_Id" value for requests and exceptions. This value is copied from the customDimensions object. Setting the user_Id makes it easier for creating dashboards on the Azure portal.Kishore BattulaVibhuti Sharma [Microsoft]Kishore Battulahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/97Draft: Added EntitlementsFactoryAzure in Core-Lib-Azure2021-08-25T05:38:31ZMuskan SrivastavaDraft: Added EntitlementsFactoryAzure in Core-Lib-Azure## All Submissions:
-------------------------------------
* I have added an explanation of what changes in this merge do and why we should include it? - YES
* Does the MR contain pipeline/ helm chart related changes?- NO
* I have updated...## All Submissions:
-------------------------------------
* I have added an explanation of what changes in this merge do and why we should include it? - YES
* Does the MR contain pipeline/ helm chart related changes?- NO
* I have updated the documentation accordingly.- YES
* I have added tests to cover my changes. -NA
* All new and existing tests passed.- YES
* My code follows the code style of this project. -YES
* I ran lint checks locally prior to submission.-YES
## What is the issue or story related to the change?
-------------------------------------
This is one of the MRs that will be required to add configurable Retries and Timeouts functionality to the services. It adds EntilementsFactoryAzure which will have retry and timeouts enabled for the calls made to Entitlements - Service V2. This EntitlementsFactoryAzure can be used in the services which want retry and timeouts enabled for entitlements service. It is same as "EntitlementsFactory" class of core-common in Entitlements-V2, except for the fact that it will use HttpClientAzure instead of HttpClient which will have retry and timeouts enabled.
High level design: Please refer to this [design doc](https://microsoft.sharepoint.com/:w:/t/EnergyDataPlatform/ETXMusBJg8pKvDxImoChXOYBjxI27tKCar1iG7M6V4Ut5Q?e=zEo4rd). This MR can be seen as one of the steps taken under sub-section named "[OPTION-1]- Have our own Http Client"
## Does this introduce a breaking change?
-------------------------------------
- NO
## Pending items
----------------
Following MR will be brought up:
- HttpClientAzure: implementation
- RetryAndTimeoutsConfiguration: properties for timeouts and retries configuration will be added
- EntitlementsFactoryAzure: retry and timeouts will be set for EntitlementsService class
- Other factory classes on the lines of EntitlementsFactoryAzure needs to be added.Muskan SrivastavaMuskan Srivastavahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/144Remove conditional beans2021-09-17T07:15:45ZNikhil Singh[MicroSoft]Remove conditional beansThis MR removes the conditional bean property from message broker beans because the MessagePublisher Bean uses both service bus and event grid beans.This MR removes the conditional bean property from message broker beans because the MessagePublisher Bean uses both service bus and event grid beans.Nikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/163Draft: Override vulnerable netty version <4.1.682021-11-09T16:40:57ZDmitrii GerashchenkoDraft: Override vulnerable netty version <4.1.68Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/164Draft: Upgrade vulnerable dependencies2021-11-16T14:41:13ZDmitrii GerashchenkoDraft: Upgrade vulnerable dependencieshttps://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/88
**Summary**
WhiteSource's vulnerabilities list contain alerts about:
- spring-web-5.3.6.jar
- netty-codec-4.1.63.Final.jar
- netty-codec-4.1...https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/88
**Summary**
WhiteSource's vulnerabilities list contain alerts about:
- spring-web-5.3.6.jar
- netty-codec-4.1.63.Final.jar
- netty-codec-4.1.63.Final.jar
- spring-security-oauth2-client-5.4.6.jar
- netty-all-4.1.63.Final.jar
- netty-handler-4.1.63.Final.jar
Updated:
- spring-boot.version from 2.4.[-5-] to 2.4.[+12+]
- netty-bom.version form 4.1.[-63-].Final to 4.1.[+70+].Final
**Related MRs:**
- Entitlements
- MR: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/143
- Trusted branch pipeline is passed: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/pipelines/76742
- os-core-common
- MR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/124
- Trusted branch pipeline is passed: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/pipelines/76515
- os-core-lib-azure
- MR: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/164
- Trusted branch pipeline is passed: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/pipelines/76697Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/82Added Async blob store2021-11-19T12:09:54ZNicholas KarskyAdded Async blob store## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] I have updated the documentation accordingly.
* [YES] I have added tests...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
Adding the factory files necessary to implement BlockBlobAsyncClient for blob uploads
High level design:
Introduces use of Flux and Mono in order to speed up the blob upload process
## Test coverage:
------------------
100% coverage
## Does this introduce a breaking change?
-------------------------------------
- [NO]
## Pending items
----------------
N/A
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->Nicholas KarskyNicholas Karskyhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/99Draft: added httpClientAzure, EntitlementsFactoryAzure and RetryAndTimeoutsCo...2021-11-19T12:10:02ZKomal MakkarDraft: added httpClientAzure, EntitlementsFactoryAzure and RetryAndTimeoutsConfigurationhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/84Draft: Users/komakkar/adapter implementation2021-11-19T12:10:11ZKomal MakkarDraft: Users/komakkar/adapter implementationhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/83Adapters for PartitionInfo2021-11-19T12:10:12ZKomal MakkarAdapters for PartitionInfo## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have ...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have updated the documentation accordingly.
* [NA] I have added tests to cover my changes.
* [NA] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
*Note*: The MR is not an implementation MR. Hence, please don't expect tests to be in place.
**Existing Behaviour**: The PartitionInfo is not consumable easily. PartitionInfoAzure was the means to consume it. The deserialization had hardcoded the keys of the map. This limits us to add keys on the fly and consume those.
The right-hand side is the representation of how we are consuming it today.
**Modified Behaviour**:
The left hand represents the modified behavior. We can break it down as follows to understand it better.
1. Breaking free from PartitionInfoAzure. We encourage Data Models for each resource type, for instance, EventGridPartitionConfig.
2. Adapter for creating the config. These adapters are supposed to convert PartitionInfoAzure to Config data model. For instance EventGridPartitionConfigAdapter.
3. PartitionInfoProvider. This provider is meant to make sure the infra stores have the access to PartitionInfo. PartitionInfo is supplied to the adapters.
High level design:
![image](/uploads/9344487aca80e938f91df51a7e072342/image.png)
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
#6
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [YES/NO] No
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/167Fixed log4j CVE-2021-442282021-12-14T17:12:07ZKhilesh SahuFixed log4j CVE-2021-44228Upgraded log4j-core and log4j-api to 2.15.0 as direct dependency
Default spring boot was using 2.13.0, we are upgrading to 2.15.0.
Issue link : https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/is...Upgraded log4j-core and log4j-api to 2.15.0 as direct dependency
Default spring boot was using 2.13.0, we are upgrading to 2.15.0.
Issue link : https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/13https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/106Draft: Cryptography Retry Configuration2021-12-16T16:23:10ZRonak SakhujaDraft: Cryptography Retry Configuration## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] I have updated the documentation accordingly.
* [YES] I have added tests ...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
Cryptographyoffers option for configuring various attributes for retry policy. For any service who wish not to use the default values, and want to provide custom values for the mentioned attributes, they will be able to after this change.
High level design:
![image](/uploads/5d300661ada45b326a9c0043da4f9b89/image.png)
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
Change details:
Variables that can be added to application.properties are:
azure.cryptography.retryStrategyValue
azure.cryptography.FixedDelayConfiguration.maxRetries
azure.cryptography.FixedDelayConfiguration.delay
azure.cryptography.ExponentialBackoffConfiguration.maxRetrues
azure.cryptography.ExponentialBackoffConfiguration.baseDelayInMillis
azure.cryptography.ExponentialBackoffConfiguration.maxDelayInSeconds
azure.cryptography.retryAfterHeader
azure.cryptography.retryAfterTimeUnit
## Test coverage:
------------------
All changes are covered.
## Does this introduce a breaking change?
-------------------------------------
- NO
## Pending items
----------------
Documentation will come in a separate MR.
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/130Update common.properties to enable health check bean2022-01-11T13:24:30ZSmitha ManjunathUpdate common.properties to enable health check bean## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have...## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have added tests to cover my changes.
* [YES/NO/NA] All new and existing tests passed.
* [YES/NO/NA] My code follows the code style of this project.
* [YES/NO/NA] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [YES/NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->Smitha ManjunathSmitha Manjunathhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/127Added docs - updated notice file2022-01-11T13:24:34ZSmitha ManjunathAdded docs - updated notice file## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have...## All Submissions:
-------------------------------------
* [YES/NO] I have added an explanation of what changes in this merge do and why we should include it?
* [YES/NO] I have updated the documentation accordingly.
* [YES/NO/NA] I have added tests to cover my changes.
* [YES/NO/NA] All new and existing tests passed.
* [YES/NO/NA] My code follows the code style of this project.
* [YES/NO/NA] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [YES/NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->Smitha ManjunathSmitha Manjunathhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/123Draft: Made changes for adding cache staleness for the new version of core-li...2022-01-11T13:24:34ZAbhiram BondadaDraft: Made changes for adding cache staleness for the new version of core-lib-azure