ADR: Python: Community implementation versions of artifacts
ADR
Title
Dynamically Loading the Blob Storage Client and Credentials Implementations
Context
The community implementation requires maintaining only the core logic within the shared repository. Specific cloud-level implementations (e.g., AWS, Azure, Google) must be developed separately while adhering to the core logic. This ensures flexibility and interoperability across various cloud platforms.
Decision
We have opted for a dynamic module import strategy leveraging Python's importlib package. The MR for Python SDK is here. This allows each cloud provider to write their own implementations and install them alongside the basic Python SDK.
The implementations would align with abstract classes defined in the python-sdk-abstract-classes repository.
For instance, Google Cloud's requirements.txt
may look as follows:
--extra-index-url=https://community.opengroup.org/api/v4/projects/1460/packages/pypi/simple
osdu-api-abstract
--extra-index-url=https://community.opengroup.org/api/v4/projects/148/packages/pypi/simple
osdu-api==0.25.0.dev804
--extra-index-url=https://community.opengroup.org/api/v4/projects/1461/packages/pypi/simple
osdu-api-gc==0.25.0.dev9
The paths to the specific implementations can be defined in two environment variables: PYTHON_SDK_CREDENTIALS_MODULE
and PYTHON_SDK_BLOB_STORAGE_MODULE
. These point to the get_credentials
and get_blob_storage
functions respectively, which return concrete implementations of the BaseCredentials
and BlobStorageClient
abstract classes. See the following example: get_blob_storage and get_credentials
In our Google Cloud environment, the environment variables would look like:
# .env file
PYTHON_SDK_CREDENTIALS_MODULE="osdu_api_gc.credentials"
PYTHON_SDK_BLOB_STORAGE_MODULE="osdu_api_gc.blob_storage"
Class Diagrams
Consequences
The dynamic module loading approach offers enhanced flexibility and interoperability across various cloud platforms. It isolates dependencies specific to each cloud provider, ensuring their implementations are developed and installed separately, mitigating any dependency conflicts. The approach calls for strict adherence to defined abstract classes in implementations, facilitating a consistent interface across different modules. However, comprehensive testing and validation are required to ensure custom implementations align with the defined abstract classes, ensuring smooth interaction with the core logic.
Also, unlike Java services, Python services don't use a shared code base. Services to wrk on: