Python SDK merge requestshttps://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests2023-08-18T14:47:13Zhttps://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/58Create config from environment variables in Azure2023-08-18T14:47:13Zharshit aggarwalCreate config from environment variables in Azure## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M10 - Release 0.13https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/57feat(azure): add request raiser decorator2023-08-18T14:47:14ZVadzim Kulybafeat(azure): add request raiser decorator## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] ...## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
This functionality is needed for (pipeline failed without this module):
https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/issues/3
https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/11/diffsM10 - Release 0.13Vadzim KulybaVadzim Kulybahttps://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/56feat(azure): update azure credentials service2023-08-18T14:47:16ZVadzim Kulybafeat(azure): update azure credentials service## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] ...## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Refactor azure credentials service (more friendly API, not only for airflow usage)
https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/issues/9M10 - Release 0.13Vadzim KulybaVadzim Kulybahttps://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/55feat(dataset_dms): add new retrieval instruction aproach2023-08-18T14:47:17ZVadzim Kulybafeat(dataset_dms): add new retrieval instruction aproach## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] ...## Type of change
- [] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [x] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Added new retrieval Instructions request method for dataset client, because we have several approaches on getting retrieval instructions for datasets
Details:
https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/issues/10M10 - Release 0.13Vadzim KulybaVadzim Kulybahttps://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/53GONRG-3783: Common pipeline for osdu-*2022-09-16T08:17:26ZYan Sushchynski (EPAM)GONRG-3783: Common pipeline for osdu-*## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Add common pipelines.M10 - Release 0.13Vladislav Shishko (EPAM)Vladislav Shishko (EPAM)https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/52revert data-workflow-client2022-09-16T08:15:54ZYan Sushchynski (EPAM)revert data-workflow-client## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
`data_workflow_client` was mistakenly considered the main workflow_client instead of `ingestion_workflow_client`.
This MR reverts changes of this client and the corresponding modelM10 - Release 0.13Siarhei Khaletski (EPAM)Siarhei Khaletski (EPAM)https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/51Removing bad lib2023-08-18T14:47:19ZSpencer Suttonsuttonsp@amazon.comRemoving bad lib## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Describe your code changes in details for reviewers (links on Gitlab issues, etc.)M10 - Release 0.13Spencer Suttonsuttonsp@amazon.comSpencer Suttonsuttonsp@amazon.comhttps://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/50GONRG-3618: Local build setup2022-09-16T08:14:45ZYan Sushchynski (EPAM)GONRG-3618: Local build setup## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ]...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Updates description?
Resolves the issue: https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/issues/1, when it was needed to specify extra environmental variables if the package was built locally.M10 - Release 0.13Siarhei Khaletski (EPAM)Siarhei Khaletski (EPAM)https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/49GONRG-3119: Python SDK usage2022-09-16T08:12:57ZYan Sushchynski (EPAM)GONRG-3119: Python SDK usage## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [x] Azure
- [x] GCP
- [x]...## Type of change
- [ ] Bug Fix
- [x] Feature
## Does this introduce a change in the core logic?
- [Yes]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [x] Azure
- [x] GCP
- [x] IBM
## Updates description?
Added `TokenRefresher` mechanism to obtain `access_token` in `BaseClient` for using Python SDK's clients in Ingestion Lib instead of `requests` lib.M10 - Release 0.13Siarhei Khaletski (EPAM)Siarhei Khaletski (EPAM)