1. This will be the OSDU Data loader utility to support Wellbore DDMS.
2. This is a wrapper behind the Wellbore DDMS [API](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/master/spec/generated/openapi.json)
## Install from Package Registry
### Before installation:
- Ensure Python 3.8 or newer is installed and that Python has been added to the PATH (in your system's environment variables).
- Confirm you have `pip` installed by running the following command (it should be installed automatically when you installed Python) - `pip --version`
- If installing `wbdutil` within a virtual environment, first install `pipenv` using the following command - `pip install pipenv`
- Note: There is a bug in `pipenv` version 2021.11.23 which will cause the `wbdutil` installation command to fail. Ensure your version number is different to this (e.g. `pip install pipenv=="2021.11.15"` or alternatively a newer version).
- Ensure Python 3.8 or newer is installed and that Python has been added to the PATH (in your system's environment variables).
- Confirm you have `pip` installed by running the following command (it should be installed automatically when you installed Python) - `pip --version`
- If installing `wbdutil` within a virtual environment, first install `pipenv` using the following command - `pip install pipenv`
- Note: There is a bug in `pipenv` version 2021.11.23 which will cause the `wbdutil` installation command to fail. Ensure your version number is different to this (e.g. `pip install pipenv=="2021.11.15"` or alternatively a newer version).
### Installation:
The simplest way to install `wbdutil` is from the community package registry.
**Method 1:** To download and install `wbdutil` on your machine run:
**Method 2:** Alternatively, you may install the `wbdutil` package within a virtual environment using `pipenv`. If you have `pipenv` installed, simply run:
@@ -85,7 +88,7 @@ The `wbdutil` requires a configuration file that has the following JSON structur
}
```
The `base_url` and `data_partition_id` must be correct for the OSDU instance that you want to connect to.
The `base_url` and `data_partition_id` must be correct for the OSDU instance that you want to connect to.
`wellbore_mapping` and `welllog_mapping` are optional features described in the [custom mappings](#custom-mappings) section.
#### Custom mappings
...
...
@@ -99,7 +102,7 @@ There are 3 mapping definitions `wellbore_mapping`, `welllog_mapping` and `las_f
The first 2 (`wellbore_mapping` and `welllog_mapping`) define mappings from LAS format data to OSDU wellbore and welllog objects.
`las_file_mapping` defines the mapping from OSDU well log, well bore and curve data to LAS format data (a `lasio.LASFile` object).
All the mapping definitions must contain a `mapping` attribute, in addition the LAS to OSDU mapping definitions (`wellbore_mapping` and `welllog_mapping`) must contain a `kind` attribute.
All the mapping definitions must contain a `mapping` attribute, in addition the LAS to OSDU mapping definitions (`wellbore_mapping` and `welllog_mapping`) must contain a `kind` attribute.
If `wellbore_mapping`, `welllog_mapping` and `las_file_mapping` are not defined in the configuration file `wbdutil` will use the default mappings.
The `mapping` attribute describes how data in the incoming object should be transformed into the outgoing data type.
The `kind` attribute defines the target OSDU data type (kind), for example `osdu:wks:work-product-component--WellLog:1.1.0`.
...
...
@@ -143,7 +146,7 @@ Here is an example mapping for a welllog that could be added to a configuration
}
```
The simple data mappings take the form of a key and string value pair.
The simple data mappings take the form of a key and string value pair.
The key (string to the left of the semi-colon) is the target field within the OSDU data kind and
the value string defines the source of the data. For example:
`"data.ReferenceCurveID": "curves[0].mnemonic"` will set the `data.ReferenceCurveID` field of the output OSDU object
...
...
@@ -157,6 +160,7 @@ There are often more complex transformations that need to be performed on the in
`wbdutil` supports two types of complex mapping `array` and `function`.
The `function` complex mapping type makes a call to a hard coded function to perform a transformation on the incoming data.
For example:
```
"CurveUnit": {
"type": "function",
...
...
@@ -167,12 +171,14 @@ For example:
]
}
```
This will set the value of the `CurveUnit` field to the output of the function `las2osdu_curve_uom_converter` using the input arguments in the args array. The `args` section defines the argument for the function, each `arg` is not the direct input argument to the function. An `arg` is a reference to piece of data in the incoming data or configuration file.
In this case `unit` references data in the input data and `data_partition_id` data in the configuration file.
In this case `unit` references data in the input data and `data_partition_id` data in the configuration file.
The second complex mapping is `array` this should be used if the elements of an incoming array need to be changed in some way.
This could be a field name change, a change in the object structure or to call a function on specific data within each element.
Here is an example:
```
{
"data.Curves": {
...
...
@@ -193,15 +199,17 @@ Here is an example:
}
}
```
This mapping will iterate over the `curves` array of the input `lasio.LASFile` object and apply an
inner mapping to each element in the array.
In this case the inner mapping is defined so that the `mnemonic` field of the `curve` element is
mapped to both the `CurveID` and `Mnemonic` output fields, and the `CurveUnit` output field is set
to the return value of the function `las2osdu_curve_uom_converter`
to the return value of the function `las2osdu_curve_uom_converter`
that takes the `unit` field of array element and the `data_partition_id` (from configuration) as arguments.
The resulting output array is mapped to the `data.Curves` field of the output OSDU kind.
Here is an example `las_file_mapping` section:
```
"las_file_mapping": {
"mapping": {
...
...
@@ -219,6 +227,7 @@ Here is an example `las_file_mapping` section:
}
}
```
With OSDU to LAS mappings, data is drawn from 3 types of OSDU data object: wellbore, welllog and curves.
Each of these incoming OSDU data can be referenced by the keywords `WELLBORE`, `WELLLOG` and `CURVES`.
Where `WELLBORE` and `WELLLOG` are wellbore and welllog OSDU kinds and `CURVES` is a Pandas DataFrame that
...
...
@@ -229,21 +238,21 @@ it also contains example custom mappings for the `osdu:wks:master-data--Wellbore
This table summarises the available keywords.
| Keyword | Valid Mapping type | Incomming data source |
| `build_wellbore_name_aliases(uwi, data_partition_id)` | `wellbore_mapping` | Constructs a name alias object from the LAS UWI and the data partition id. |
| `get_wellbore_id()` | `welllog_mapping` | Returns the wellbore id from the wellbore that corresponds to the welllog |
| `las2osdu_curve_uom_converter(unit, data_partition_id)` | `welllog_mapping` | This function converts a LAS format unit of measure to an OSDU format UoM. |
| `extract_uwi_from_name_aliases(NameAliases: list)` | `las_file_mapping` | Return the first name alias or None if none exist |
| `build_wellbore_name_aliases(uwi, data_partition_id)` | `wellbore_mapping` | Constructs a name alias object from the LAS UWI and the data partition id. |
| `get_wellbore_id()` | `welllog_mapping` | Returns the wellbore id from the wellbore that corresponds to the welllog |
| `las2osdu_curve_uom_converter(unit, data_partition_id)` | `welllog_mapping` | This function converts a LAS format unit of measure to an OSDU format UoM. |
| `extract_uwi_from_name_aliases(NameAliases: list)` | `las_file_mapping` | Return the first name alias or None if none exist |
| `build_curves_section(wl_curves: list, curves: DataFrame)` | `las_file_mapping` | Iterates over curves, converting units of measure from OSDU to LAS form. Returns the updated curve data. |
These are hard coded functions, so a change request will need to be raised if additional functions are required. We have avoided user defined functions, because such functions represent a small security risk.
