OS Core Lib Azure merge requestshttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests2022-01-18T19:23:41Zhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/176Resolve "Upgrade to Log4J 2.17.1 to address CVE-2021-44832"2022-01-18T19:23:41ZDavid Diederichd.diederich@opengroup.orgResolve "Upgrade to Log4J 2.17.1 to address CVE-2021-44832"Closes #16Closes #16M10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/175Data lake store2023-08-18T12:43:33ZHarshit SaxenaData lake storeIn these changes, I am adding the Azure DataLake Gen 2 store which will be used by the file collection feature.
Changes -
1) create directory in Azure DataLake file System
2) generate signed URL for Azure DataLake file System's directory.In these changes, I am adding the Azure DataLake Gen 2 store which will be used by the file collection feature.
Changes -
1) create directory in Azure DataLake file System
2) generate signed URL for Azure DataLake file System's directory.M10 - Release 0.13Harshit SaxenaHarshit Saxenahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/174Cherry-pick log4j updates into the release branch2021-12-20T20:26:57ZDavid Diederichd.diederich@opengroup.orgCherry-pick log4j updates into the release branchOriginal MR: !173
Part of the #15 seriesOriginal MR: !173
Part of the #15 seriesDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/173Update dependencies - log4j2023-08-18T12:43:34ZAlok JoshiUpdate dependencies - log4jPart of the #15 seriesPart of the #15 seriesM10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/171Adding file collection storage account2023-08-18T12:43:36ZHarshit SaxenaAdding file collection storage accountTo support the file collection feature in file service, a storage account with hierarchical namespace is required. Azure Data Lake Gen 2 has this capability. Below mentioned changes supports accessing the account name and key from partit...To support the file collection feature in file service, a storage account with hierarchical namespace is required. Azure Data Lake Gen 2 has this capability. Below mentioned changes supports accessing the account name and key from partition service.M10 - Release 0.13Harshit SaxenaHarshit Saxenahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/170Cherry-pick 'Update log4j again' into 'release/0.12'2022-12-22T18:58:03ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Update log4j again' into 'release/0.12'Original MR: !169
(cherry picked from commit 74912ef44f447019905a5f65e660345335afac63)
Part of the #14 seriesOriginal MR: !169
(cherry picked from commit 74912ef44f447019905a5f65e660345335afac63)
Part of the #14 seriesDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/169Update log4j again2023-08-18T12:43:38ZDavid Diederichd.diederich@opengroup.orgUpdate log4j againUpdating log4j to 2.16 to address [CVE-2021-45046](https://nvd.nist.gov/vuln/detail/CVE-2021-45046)
Part of the #14 seriesUpdating log4j to 2.16 to address [CVE-2021-45046](https://nvd.nist.gov/vuln/detail/CVE-2021-45046)
Part of the #14 seriesM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/168Cherry-pick branch 'Upgrade vulnerable dependencies according to WhiteSource ...2021-12-15T00:16:14ZDavid Diederichd.diederich@opengroup.orgCherry-pick branch 'Upgrade vulnerable dependencies according to WhiteSource alerts' into 'release/0.12'Original MR: !165
Part of the #14 series
(cherry picked from commit f4f7110f8153bca5ca5175aad4db1ad704644ede)Original MR: !165
Part of the #14 series
(cherry picked from commit f4f7110f8153bca5ca5175aad4db1ad704644ede)David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/166handle request too large exception2023-08-18T12:43:40ZAlok Joshihandle request too large exceptionPlease refer to https://community.opengroup.org/osdu/platform/system/storage/-/issues/97
Making this change to handle the particular exception in exception mapper so it is handled generically.Please refer to https://community.opengroup.org/osdu/platform/system/storage/-/issues/97
Making this change to handle the particular exception in exception mapper so it is handled generically.M10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/165Upgrade vulnerable dependencies according to WhiteSource alerts2023-08-18T12:43:41ZDmitrii GerashchenkoUpgrade vulnerable dependencies according to WhiteSource alertshttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/52
---
os-core-common version upgrade after MR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/126
---
*...https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/52
---
os-core-common version upgrade after MR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/126
---
**os-core-common** was updated according to WhiteSource alerts:
- spring-boot.version: 2.4.[-5-] -> 2.4.[+12+]
- netty-bom.version: 4.1.[-63-].Final -> 4.1.[+70+].Final
- json-smart.version: 2.4.[+7+]
- gson.version: 2.8.[-5-] -> 2.8.[+9+]
---
json-smart was removed from **os-core-lib-azure** pom because the correct version of this dependency is provided by **os-core-common** after the upgrade.
---
**The list of successfully tested related services which will use the upgraded version (there are no significant issues after the upgrade):**
- **os-core-lib-azure:** https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/165
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/pipelines/78303
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/82880ba928b00ea4c818be51f85b98c3/image.png)</details>
- **entitlements:** https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/149
- Successful pipeline: https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/pipelines/78617
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/dbc61f0c520940573216fadc95e8139b/image.png)</details>
- **partition:** https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/117
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/partition/-/pipelines/78619
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/d91211a3c37afc4c67ccae470f2d25f0/image.png)</details>
- **unit-service:** https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/115
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/pipelines/78626
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/7926b86b814facd4c73e4ea8193d9dab/image.png)</details>
- **crs-conversion-service:** https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/90
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/pipelines/78526
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/61042606d00ce9a25ae9031e6c7b9807/image.png)</details>
- **crs-catalog-service:** https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/78
- Successful pipeline: https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/pipelines/78351
- <details><summary>screenshot</summary>![image](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/uploads/163f01078184ea08263abbcb4cd4da50/image.png)</details>
Part of the #14 seriesM10 - Release 0.13Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/162Size limit2023-08-18T12:43:43ZAlok JoshiSize limitMinor improvement over https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/160Minor improvement over https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/160M10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/161Update notice2023-08-18T12:43:45ZAlok JoshiUpdate noticeFix notice fileFix notice fileM10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/160Catch request too large exception2023-08-18T12:43:46ZAlok JoshiCatch request too large exceptionPlease refer to https://community.opengroup.org/osdu/platform/system/storage/-/issues/97 for detailsPlease refer to https://community.opengroup.org/osdu/platform/system/storage/-/issues/97 for detailsM10 - Release 0.13Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/159Cherry-pick branch 'users/nikhil/fixHttpClientRS' into release/0.122022-12-22T18:58:01ZDavid Diederichd.diederich@opengroup.orgCherry-pick branch 'users/nikhil/fixHttpClientRS' into release/0.12Make HttpClientAzure conditionally primary
See original merge request osdu/platform/system/lib/cloud/azure/os-core-lib-azure!158Make HttpClientAzure conditionally primary
See original merge request osdu/platform/system/lib/cloud/azure/os-core-lib-azure!158M9 - Release 0.12David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/158Make HttpClientAzure conditionally primary2023-08-18T12:43:48ZNikhil Singh[MicroSoft]Make HttpClientAzure conditionally primary** Introduction **
This MR makes the bean HttpClientAzure Conditional so that worker services like Indexer Queue,WKS,Notification,Storage can use the non request scope HttpClient while working with message bus.** Introduction **
This MR makes the bean HttpClientAzure Conditional so that worker services like Indexer Queue,WKS,Notification,Storage can use the non request scope HttpClient while working with message bus.M9 - Release 0.12Nikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/157Update OSDU dependencies2021-10-21T16:10:34ZDavid Diederichd.diederich@opengroup.orgUpdate OSDU dependenciesM9 - Release 0.12David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/156Update core common version2023-08-18T12:43:50ZNikhil Singh[MicroSoft]Update core common version**Introduction**
This MR upgrades the os core-common version for azure provider to verify the recent security fixes made in the below mentioned MR are working properly.
**Issue Link**
https://community.opengroup.org/osdu/platform/syst...**Introduction**
This MR upgrades the os core-common version for azure provider to verify the recent security fixes made in the below mentioned MR are working properly.
**Issue Link**
https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/119M9 - Release 0.12Nikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/155throw explicit 503 when server is busy to process BlobStorage request so clie...2023-08-18T12:43:51ZNeelesh Thakurthrow explicit 503 when server is busy to process BlobStorage request so client can retryWe see Storage API starts to responds with 500 when system is under heavy ingestion for very few requests. On closer look, we found that BlobStorage is throttled and underlaying error message is following:
AppException(error=AppError(co...We see Storage API starts to responds with 500 when system is under heavy ingestion for very few requests. On closer look, we found that BlobStorage is throttled and underlaying error message is following:
AppException(error=AppError(code=500, reason=Failed to read specified blob, message=Status code 503, "<?xml version="1.0" encoding="utf-8"?><Error><Code>ServerBusy</Code><Message>The server is busy._RequestId:82895e12-701e-0066-26d3-b568b7000000_Time:2021-09-30T08:13:49.5116925Z</Message></Error>", errors=null, debuggingInfo=null, originalException=com.azure.storage.blob.models.BlobStorageException: Status code 503, "<?xml version="1.0" encoding="utf-8"?><Error><Code>ServerBusy</Code><Message>The server is busy.
RequestId:82895e12-701e-0066-26d3-b568b7000000
Time:2021-09-30T08:13:49.5116925Z</Message></Error>"), originalException=com.azure.storage.blob.models.BlobStorageException: Status code 503, "<?xml version="1.0" encoding="utf-8"?><Error><Code>ServerBusy</Code><Message>The server is busy.
RequestId:82895e12-701e-0066-26d3-b568b7000000
Time:2021-09-30T08:13:49.5116925Z</Message></Error>")
![image](/uploads/2ac4650821f1a3ae252b65f4a5958d3c/image.png)
With this change, client can retry based of response code.M9 - Release 0.12https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/154Update common.properties2023-08-18T12:43:53ZSmitha ManjunathUpdate common.propertiesM9 - Release 0.12Smitha ManjunathSmitha Manjunathhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/153Added code to disable ai appender based on jvm argument2023-08-18T12:43:55ZVineeth Guna [Microsoft]Added code to disable ai appender based on jvm argumentM9 - Release 0.12