Feature Request: Python Async SDK

ADR: Python Async Client Library for OSDU

Status

  • Proposed
  • Trialing
  • Under review
  • Approved
  • Retired

Context & Scope

Currently, there are Python services within the OSDU ecosystem communicate with the OSDU platform. These services independently implement the same logic for making API calls (e.g., Wellbore DDMS and RAFS DDMS). This practice of re-implementing code creates several problems: it is inefficient, leads to inconsistencies between services, and makes maintenance difficult.

Java-based services have already solved this issue by using a shared library, the os-core-common client library, which provides a single, reliable way to interact with OSDU services.

In the Python ecosystem, a common-python-sdk exists, but it is not ergonomic for modern development and lacks support for modern asynchronous programming.

This situation creates a clear need to standardize and simplify how current and future Python services connect to the OSDU platform. The scope of this decision applies to all Python services that need to interact with OSDU APIs.

Decision

A new, modern Python Async Client Library must be developed. This library will become the single, recommended tool for all Python services to interact with the OSDU API.

The new client library will be built using a modern technology stack, including:

  • Asyncio: For using in modern web frameworks and applications (e.g., FastAPI).
  • Pydantic: For strict data validation and clear data models that match the OSDU API schemas.
  • OpenAPI Code Generation: Utilizing OpenAPI code generation to automatically create data models from existing OpenAPI specifications (e.g., this tool).
  • Static Typing: To improve code quality, enhance developer experience, and reduce bugs.
  • uv: A fast and modern package manager to streamline dependency management for the library itself.
  • pytest: A modern framework for testing Python code.
  • CI/CD Automation: A fully automated Continuous Integration and Continuous Deployment pipeline for linting, testing, building, versioning, and publishing the library. This ensures that the development and release lifecycle is automated as much as possible to maintain high quality and consistency.

Rationale

Adopting a centralized client library provides several key benefits:

  • Eliminates Code Duplication: Developers will no longer need to write boilerplate code for API calls. They can import the library and immediately focus on business logic.
  • Enforces Consistency: All services will use the same methods and models to communicate with OSDU, ensuring uniform behavior.
  • Simplifies Maintenance: When the OSDU API changes, only the client library needs to be updated. Consumer services will only need to update the library version, which is much faster and safer than modifying code in multiple repositories.
  • Improves Developer Experience: The library will provide a clean, modern, and well-documented interface. Using tools like Pydantic and static typing will make development faster and more reliable.
  • Can be used for API testing: Since the models are generated from OpenAPI specifications, the client library can be used as a reliable client for API integration and end-to-end testing.

Consequences

  • Initial Investment: There will be an upfront effort to design, build, test, and document the first version of the client library.
  • Adoption Effort: Existing Python services will need to be refactored to remove their custom API logic and adopt the new library.
  • Long-Term Gain: After the initial investment, the speed of development for new features and services will increase significantly. Maintenance costs related to API changes will be greatly reduced.
  • New Infrastructure: A dedicated repository and a CI/CD pipeline will be created to manage the library's lifecycle, including testing, versioning, and publishing.

Tradeoff Analysis - Input to decision

Currently, individual teams can move quickly on their own, but this creates duplicated code and future maintenance challenges. This ADR is based on the observation that the long-term cost of maintaining multiple, inconsistent API clients is higher than the one-time cost of building and adopting a single, shared client.

Alternatives and implications

  • Alternative 1: Continue with the current approach.
    • Implication: The Community will continue to spend development time on rewriting the same code. As the number of services and APIs grows, the cost of maintenance will increase, and the risk of inconsistencies will rise.
  • Alternative 2: Modernize the existing common-python-sdk.
    • Implication: The existing SDK is not built with a modern, asynchronous approach in mind. Refactoring it to meet our standards would likely require a complete rewrite, making it more efficient to start fresh with a clear vision and modern tools.

Decision criteria and tradeoffs

The decision to build a new client library was based on the following criteria:

  • Developer Efficiency: Prioritizing a solution that allows developers to focus on business value, not boilerplate code.
  • Long-Term Maintainability: Choosing a path that simplifies updates and reduces future technical debt.
  • Code Quality and Reliability: Ensuring that our interactions with the OSDU platform are consistent and robust.

The main tradeoff is accepting an initial period of investment (building the library and refactoring services) in exchange for significant long-term benefits in speed, quality, and maintainability. We are choosing the path that provides the highest long-term value.

Decision timeline

  • ADR Proposal Date: 2025-11-27
  • Review Period: 2025-11-27 - 2025-12-26
  • Final Decision Date: 2025-12-27
  • Target Implementation Start: 2026-04-27
Edited by Yan Sushchynski [EPAM/GC]
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information