OS Core Lib Azure merge requestshttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests2023-08-18T12:43:19Zhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/193Standardize worker logs2023-08-18T12:43:19ZVibhuti Sharma [Microsoft]Standardize worker logs**Aim**
Enable worker services to follow a common logging format.
**Implementation**
* Added an Abstract class for MessageHandler class that is defined in worker services like WKS and Indexer Queue, to follow a common start and end work...**Aim**
Enable worker services to follow a common logging format.
**Implementation**
* Added an Abstract class for MessageHandler class that is defined in worker services like WKS and Indexer Queue, to follow a common start and end worker log template.
* Added WorkerPayload class in logging module, along with logWorker method in CoreLogger
**Related MRs**
* https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/382/diffsM11 - Release 0.14Vibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/190Adding MSI Authentication for Azure Resource2023-08-18T12:43:21Zharshit aggarwalAdding MSI Authentication for Azure ResourceM11 - Release 0.14harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/188increased version for storage blob2023-08-18T12:43:22ZAnkur Rawatincreased version for storage blobFluxutil is available in azure-core package for version >= 1.17.0Fluxutil is available in azure-core package for version >= 1.17.0M11 - Release 0.14Ankur RawatAnkur Rawathttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/187Fix Notice2023-08-18T12:43:24Zharshit aggarwalFix NoticeM11 - Release 0.14harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/186Add option to use MSI for Authentication to Azure Resources2023-08-18T12:43:26Zharshit aggarwalAdd option to use MSI for Authentication to Azure ResourcesChanges in the MR are focused on adding option to use Managed Identity Credentials while authenticating to Azure resourcesChanges in the MR are focused on adding option to use Managed Identity Credentials while authenticating to Azure resourcesM11 - Release 0.14harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/183Make DataLakeProvider a conditional bean2023-08-18T12:43:27ZRostislav Vatolinvatolinrp@gmail.comMake DataLakeProvider a conditional beanFixes https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/19Fixes https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/19M11 - Release 0.14https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/180Removing unused variables2023-08-18T12:43:29ZAbhishek Kumar (SLB)Removing unused variablesFix for the issue described here: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/17Fix for the issue described here: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/17M11 - Release 0.14Abhishek Kumar (SLB)Paresh BehedeAbhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/178Created wrapper for Redis to de-couple availability of redis with service hea...2023-08-18T12:43:31ZKrishna Nikhil VedurumudiCreated wrapper for Redis to de-couple availability of redis with service health.## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] I have updated the documentation accordingly.
* [YES/NO/NA] I have added...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] I have updated the documentation accordingly.
* [YES/NO/NA] I have added tests to cover my changes. - TBD - Opened up for initial review
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
Created a wrapper for RedisCache `RedisAzureCache` that would be wired up the services even without availability of the Redis cache infrastructure.
This enables us to de-couple Redis availability with Service health. i.e., Redis unavailability will cause service degradation rather than outage.
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->M11 - Release 0.14Krishna Nikhil VedurumudiKrishna Nikhil Vedurumudihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/184Cherry-pick 'Make DataLakeProvider a conditional bean' into 'release/0.13'2022-12-22T18:58:03ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Make DataLakeProvider a conditional bean' into 'release/0.13'Original MR: !183Original MR: !183M10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/182Upgrading core library dependencies2022-01-25T01:08:43ZDavid Diederichd.diederich@opengroup.orgUpgrading core library dependenciesM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/181Revert back to 0.13.0-SNAPSHOT2022-01-24T16:15:44ZDavid Diederichd.diederich@opengroup.orgRevert back to 0.13.0-SNAPSHOTThe 0.13.1-SNAPSHOT artifact version is used by the release/0.13 branch to prepare for the first patch of 0.13 (that is,
0.13.1). Utilizing the version here on the default branch will create packages that could interfere with that develo...The 0.13.1-SNAPSHOT artifact version is used by the release/0.13 branch to prepare for the first patch of 0.13 (that is,
0.13.1). Utilizing the version here on the default branch will create packages that could interfere with that development line when
it comes into existence.
Additionally, other logic that attempts to find the latest version will return 0.13.1-SNAPSHOT as being newer than 0.13.0 (full
release), even though this isn't true. When used in the release branch, this relationship does hold -- development in preparation
for the first patch _is_ newer than the last release.M10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/179Remove extra curly brace on dependency.2022-01-21T11:55:37ZDavid Diederichd.diederich@opengroup.orgRemove extra curly brace on dependency.This was most likely a simple typoThis was most likely a simple typoM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/176Resolve "Upgrade to Log4J 2.17.1 to address CVE-2021-44832"2022-01-18T19:23:41ZDavid Diederichd.diederich@opengroup.orgResolve "Upgrade to Log4J 2.17.1 to address CVE-2021-44832"Closes #16Closes #16M10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/175Data lake store2023-08-18T12:43:33ZHarshit SaxenaData lake storeIn these changes, I am adding the Azure DataLake Gen 2 store which will be used by the file collection feature.
Changes -
1) create directory in Azure DataLake file System
2) generate signed URL for Azure DataLake file System's directory.In these changes, I am adding the Azure DataLake Gen 2 store which will be used by the file collection feature.
Changes -
1) create directory in Azure DataLake file System
2) generate signed URL for Azure DataLake file System's directory.M10 - Release 0.13Harshit SaxenaHarshit Saxenahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/173Update dependencies - log4j2023-08-18T12:43:34ZAlok JoshiUpdate dependencies - log4jPart of the #15 seriesPart of the #15 seriesM10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/171Adding file collection storage account2023-08-18T12:43:36ZHarshit SaxenaAdding file collection storage accountTo support the file collection feature in file service, a storage account with hierarchical namespace is required. Azure Data Lake Gen 2 has this capability. Below mentioned changes supports accessing the account name and key from partit...To support the file collection feature in file service, a storage account with hierarchical namespace is required. Azure Data Lake Gen 2 has this capability. Below mentioned changes supports accessing the account name and key from partition service.M10 - Release 0.13Harshit SaxenaHarshit Saxenahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/169Update log4j again2023-08-18T12:43:38ZDavid Diederichd.diederich@opengroup.orgUpdate log4j againUpdating log4j to 2.16 to address [CVE-2021-45046](https://nvd.nist.gov/vuln/detail/CVE-2021-45046)
Part of the #14 seriesUpdating log4j to 2.16 to address [CVE-2021-45046](https://nvd.nist.gov/vuln/detail/CVE-2021-45046)
Part of the #14 seriesM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/166handle request too large exception2023-08-18T12:43:40ZAlok Joshihandle request too large exceptionPlease refer to https://community.opengroup.org/osdu/platform/system/storage/-/issues/97
Making this change to handle the particular exception in exception mapper so it is handled generically.Please refer to https://community.opengroup.org/osdu/platform/system/storage/-/issues/97
Making this change to handle the particular exception in exception mapper so it is handled generically.M10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/165Upgrade vulnerable dependencies according to WhiteSource alerts2023-08-18T12:43:41ZDmitrii GerashchenkoUpgrade vulnerable dependencies according to WhiteSource alertshttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/52
---
os-core-common version upgrade after MR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/126
---
*...https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/52
---
os-core-common version upgrade after MR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/126
---
**os-core-common** was updated according to WhiteSource alerts:
- spring-boot.version: 2.4.[-5-] -> 2.4.[+12+]
- netty-bom.version: 4.1.[-63-].Final -> 4.1.[+70+].Final
- json-smart.version: 2.4.[+7+]
- gson.version: 2.8.[-5-] -> 2.8.[+9+]
---
json-smart was removed from **os-core-lib-azure** pom because the correct version of this dependency is provided by **os-core-common** after the upgrade.
---
**The list of successfully tested related services which will use the upgraded version (there are no significant issues after the upgrade):**
- **os-core-lib-azure:** https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/165
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/pipelines/78303
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/82880ba928b00ea4c818be51f85b98c3/image.png)</details>
- **entitlements:** https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/149
- Successful pipeline: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/pipelines/78617
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/dbc61f0c520940573216fadc95e8139b/image.png)</details>
- **partition:** https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/117
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/partition/-/pipelines/78619
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/d91211a3c37afc4c67ccae470f2d25f0/image.png)</details>
- **unit-service:** https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/115
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/pipelines/78626
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/7926b86b814facd4c73e4ea8193d9dab/image.png)</details>
- **crs-conversion-service:** https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/90
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/pipelines/78526
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/61042606d00ce9a25ae9031e6c7b9807/image.png)</details>
- **crs-catalog-service:** https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/78
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/pipelines/78351
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/163f01078184ea08263abbcb4cd4da50/image.png)</details>
Part of the #14 seriesM10 - Release 0.13Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/162Size limit2023-08-18T12:43:43ZAlok JoshiSize limitMinor improvement over https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/160Minor improvement over https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/160M10 - Release 0.13Alok JoshiAlok Joshi