Wellbore-azure-lib merge requestshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests2023-08-17T22:28:51Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/43Rojuneja fix storage account name2023-08-17T22:28:51ZRomica JunejaRojuneja fix storage account nameM15 - Release 0.18Romica JunejaRomica Junejahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/42removed sas token implementation2023-08-17T22:28:53ZRomica Junejaremoved sas token implementationM15 - Release 0.18Romica JunejaRomica Junejahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/40Fix creation of AzureAioBlobStorage by using legacy account_storage_name if ...2023-08-17T22:28:54ZCyril MonmoutonFix creation of AzureAioBlobStorage by using legacy account_storage_name if storage_account_blob_endpoint not availableMove back responsibility to create account_url to AzureAioBlobStorage class. And ensure backward compatibility to retrieve legacy account url if new account_blob_endpoint is not availableMove back responsibility to create account_url to AzureAioBlobStorage class. And ensure backward compatibility to retrieve legacy account url if new account_blob_endpoint is not availableM15 - Release 0.18Kin Jin NgKin Jin Nghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/36Add new RetryableDefaultAzureIdentity class to authenticate to Azure services2022-10-03T17:00:54ZCyril MonmoutonAdd new RetryableDefaultAzureIdentity class to authenticate to Azure servicesCreate new class RetryableDefaultAzureIdentity that inherits from DefaultAzureCredential with backoff policy
Note: Removing SAS token is a not part of this MR, [previous MR#34 has been reverted](https://community.opengroup.org/osdu/plat...Create new class RetryableDefaultAzureIdentity that inherits from DefaultAzureCredential with backoff policy
Note: Removing SAS token is a not part of this MR, [previous MR#34 has been reverted](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/35). It is not out of any risks and will be handled in [MR#29](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/29). cc: @Dmitriy_Rudko
related to issue #1M14 - Release 0.17Yannickfabian serinYannickhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/35Revert "Merge branch 'cmo/feature/remove-sas-token-generation' into 'master'"2023-08-17T22:28:56ZCyril MonmoutonRevert "Merge branch 'cmo/feature/remove-sas-token-generation' into 'master'"This reverts merge request !33 because it was still under review.This reverts merge request !33 because it was still under review.M14 - Release 0.17https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/34Update pipeline to run unit tests before packaging lib during Merge Requests2023-08-17T22:28:57ZCyril MonmoutonUpdate pipeline to run unit tests before packaging lib during Merge RequestsM14 - Release 0.17Dmitriy RudkoDmitriy Rudkohttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/33Implement Retryable AzureDefaultCredentials to access to blob storage2023-08-17T22:28:59ZCyril MonmoutonImplement Retryable AzureDefaultCredentials to access to blob storageRemove SAS token generation, if no credential is passed Azure Default credentials will be used by AzureBlobFileSystemWithDefaultCredentials()Remove SAS token generation, if no credential is passed Azure Default credentials will be used by AzureBlobFileSystemWithDefaultCredentials()M14 - Release 0.17https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/32Updated version to 3.0.12023-08-17T22:29:00ZDmitriy RudkoUpdated version to 3.0.1M14 - Release 0.17https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/31Storage account changes for wellbore DDMS2023-08-17T22:29:01ZVaibhavi KamaniStorage account changes for wellbore DDMS### Overview
* Merge Wellbore Storage Account with a common-one
* Made changes to point to storage account blob endpoint in case if the data partition service has `storage account blob endpoint` property else use storage account.
### Ba...### Overview
* Merge Wellbore Storage Account with a common-one
* Made changes to point to storage account blob endpoint in case if the data partition service has `storage account blob endpoint` property else use storage account.
### Backlog item
* https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_workitems/edit/13464
### Testing
* Criteria is that pipeline passed as suggested by DmitriyM14 - Release 0.17Vaibhavi KamaniVaibhavi Kamanihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/28Migrate to pytest-asyncio 0.192023-08-17T22:29:03ZDmitriy RudkoMigrate to pytest-asyncio 0.19Starting from `pytest-asyncio>=0.17`, three modes are provided: `auto`, `strict` and `legacy`. Starting from `pytest-asyncio>=0.19` the `strict` mode is the default.
In `auto` mode, the `pytest.mark.asyncio` marker can be omitted, the m...Starting from `pytest-asyncio>=0.17`, three modes are provided: `auto`, `strict` and `legacy`. Starting from `pytest-asyncio>=0.19` the `strict` mode is the default.
In `auto` mode, the `pytest.mark.asyncio` marker can be omitted, the marker is added automatically to async test functions.
See documentation:
https://pypi.org/project/pytest-asyncio/0.19.0/M14 - Release 0.17Romica JunejaRomica Junejahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/27Update python version2023-08-17T22:29:05Zfabian serinUpdate python versionMove to python 3.8 and bump major version. (Wdms moves to Python 3.8 after M10 release)Move to python 3.8 and bump major version. (Wdms moves to Python 3.8 after M10 release)M11 - Release 0.14https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/26Add tracing in Partition service2023-08-17T22:29:06ZCyril MonmoutonAdd tracing in Partition service- Use a decorator to trace with opencensus some function of the lib
- Use async version of Azure Credentials- Use a decorator to trace with opencensus some function of the lib
- Use async version of Azure CredentialsM10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/25BlobStorage-lib/Bug: upgrade to version >=12.8.1 as preconized by azure ask...2023-08-17T22:29:07ZCyril MonmoutonBlobStorage-lib/Bug: upgrade to version >=12.8.1 as preconized by azure ask...BlobStorage-lib/Bug: upgrade to version >=12.8.1 as preconized by azure ask team (https://github.com/Azure/azure-sdk-for-python/issues/19412)BlobStorage-lib/Bug: upgrade to version >=12.8.1 as preconized by azure ask team (https://github.com/Azure/azure-sdk-for-python/issues/19412)M8 - Release 0.11https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/24Dask sas token2023-08-17T22:29:09ZYannickDask sas tokenM7 - Release 0.10https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/22keep secret value to avoid multiple sync call to the keyvault2023-08-17T22:29:10ZYannickkeep secret value to avoid multiple sync call to the keyvaultBased on investigation around performance tests, get_secret is done synchronously blocking the service until the response get back (in average 90ms).
The changes here is to 'pin' the value/secret once fetched to the partitionInfo which i...Based on investigation around performance tests, get_secret is done synchronously blocking the service until the response get back (in average 90ms).
The changes here is to 'pin' the value/secret once fetched to the partitionInfo which is cached for 5 minutes. As further improvement, we could turn this call into an asynchronous one (and then optionality then the pin of secret may no longer be necessary).
In addition, an async lock has been in the fetch partition info since in multiple concurrency several income request can pass and leading to several fetch which will reset the cache and then add to synchronous the secret.M7 - Release 0.10https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/21use dash instead of forbidden underscore in container name2023-08-17T22:29:12ZYannickuse dash instead of forbidden underscore in container nameuse dash instead of forbidden underscore in container name. Azurite latest version 2021.6 now check that.use dash instead of forbidden underscore in container name. Azurite latest version 2021.6 now check that.M7 - Release 0.10https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/19logging blob storage and data patition service2023-08-17T22:29:14ZChristophe Lallementlogging blob storage and data patition servicelogging blob storage and data partition service for troubleshoot getdefaultAzureCredential and error 500 in performance testslogging blob storage and data partition service for troubleshoot getdefaultAzureCredential and error 500 in performance testsM7 - Release 0.10Christophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/18Logging2023-08-17T22:29:15ZChristophe LallementLoggingAdd logging for blob storage and partition service in order to troubleshoot performance issue and errors 500 with getDefaultAzureCredentialsAdd logging for blob storage and partition service in order to troubleshoot performance issue and errors 500 with getDefaultAzureCredentialsM7 - Release 0.10Christophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/13add dask parameters for azure2023-08-17T22:29:17ZYannickadd dask parameters for azureadd dask parameters for azureadd dask parameters for azureM7 - Release 0.10https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/lib/wellbore-cloud/wellbore-azure-lib/-/merge_requests/12Fix core version in requirement_opengroup.txt2023-08-17T22:29:19ZYannickFix core version in requirement_opengroup.txtFix core version in requirement_opengroup.txt, use core package 1.0.*Fix core version in requirement_opengroup.txt, use core package 1.0.*M5 - Release 0.8