ADR: Wellbore DDMS - Ingestion of bulk data into OSDU instance.
Introduction:
The purpose of this ADR is to address the process of importing curve data into Operator’s OSDU instance from the Provider end using the Wellbore DDMS API. The Wellbore DDMS API provides programmatic access to wellbore-related data, including curve data, allowing for efficient integration and retrieval.
Objective:
We aim to establish a direct import mechanism that enables the seamless ingestion of curve data from the provider's end into the Operator's OSDU ecosystem. By leveraging the Wellbore DDMS set of APIs, this approach will streamline the data ingestion process and enhance the capabilities of OSDU by directly incorporating valuable curve data from the provider's end.
Status
-
Proposed -
Trialing -
Under review -
Approved -
Retired
Scope
The scope of this ADR includes the following scenarios:
- Data Retrieval: Retrieval of curve data from the provider's Wellbore DDMS using the available set of Wellbore DDMS APIs.
- Ingestion Mechanism: Development and implementation of an ingestion mechanism to process and store the imported curve data.
Given /assumptions:
- Operator and Provider are supporting the wellbore DDMS functionality.
- Well log Meta data is already present at the operator end.
- This is more advanced requirement.
Required Changes:
- eds_wellbore_ddms: New API end Points will be introduced to handle the working of the DDMS wellbore.
- Below parameters will be added in the CSRE:
- DDMS URL: variable to hold the provider’s DDMS URL.
Implementation:
a. Utilize CSRE (ConnectedSource.Generic) to extract the provider details and establish a connection with the provider.
b. Initiate the wellbore DDMS URL of the provider (as provided in CSRE) with the given well log ID, retrieve the well log data and curve data at the operator's end.
c. Employ the wellbore DDMS API to ingest the curve data associated with the well log ID.
Sequence Diagram:
Functional Requirements:
- Data Retrieval:
- The set of Wellbore DDMS API should be utilized to retrieve curve data from the provider's end.
- It should support querying capabilities to fetch specific curve data.
- Data Ingestion:
- The import mechanism should provide a reliable and efficient process to ingest the curve data into the Operator’s OSDU ecosystem.
- It should handle the creation or update of curve data records within the OSDU data repositories, associating them with the appropriate wellbore entities.
Non-functional Requirements
- Performance:
- The system should be capable of handling large volumes of wellbore data efficiently, providing fast response times for data retrieval and analysis.
- It should be able to handle concurrent user interactions and maintain performance under peak load conditions.
- Scalability:
- The system should be scalable to accommodate increasing amounts of wellbore data and growing user bases.
- It should be able to handle additional data sources and support a high number of concurrent users without significant degradation in performance.