Open VDS issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues2024-02-16T16:07:49Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/228Documentation request: Unclear on constraints/expectations for channel names/...2024-02-16T16:07:49ZKevin McCartyDocumentation request: Unclear on constraints/expectations for channel names/unitsHi, this is a companion issue to #227, but here I'm asking about channels rather than axes.
* Is there a limit on the number of different "full-sized" channels (e.g. 3D for a dataset with 3 dimensions) that may be written to a VDS datas...Hi, this is a companion issue to #227, but here I'm asking about channels rather than axes.
* Is there a limit on the number of different "full-sized" channels (e.g. 3D for a dataset with 3 dimensions) that may be written to a VDS dataset?
* All the datasets that I see have "Amplitude" as the primary channel (channel 0) name. Is it legal to have a dataset where "Amplitude" is not the primary channel, or where it is not present at all?
* The datasets imported from SEGY all seem to have auxiliary "Trace" and "PDSTraceHeader" channels with per-trace values. Is it legal to create datasets that do not have those channels?
* What are the constraints on the _names_ of channels?
* Are they restricted to solely the names #define'd in `GlobalMetadataCommon.h` in section "Attributes' names"?
* Or, are they allowed to be any alphanumeric ASCII string?
* Or, are additional printable ASCII characters allowed as well (which ones?)?
* Or, is any printable string allowed? (Is the encoding required to be UTF-8 or something else?)
* What are the constraints on the _units_ of channels?
* Are they restricted to solely the names #define'd in `KnownMetadata.h` with `KNOWNMETADATA_UNIT_` prefixes / accessible from the `KnownUnitNames` class?
* Or, are they allowed to be any alphanumeric ASCII string? (And if so, is that round-trippable?)
* Etc.
As a motivation for this question, I might have need to create a channel named "Density" whose units are "g/cm^3" for instance. Or a channel named "Temperature" whose units are "degrees C", or "Salinity" with units "ppm". These are just examples off the top of my head, our own internal file formats allow for near-arbitrary channel names and units.
Thank you in advance for whatever information you can provide!https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/221Return the file size as output of the conversion of segy-vds2023-12-14T12:11:11ZDeepa KumariReturn the file size as output of the conversion of segy-vdsLinked issue: https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/issues/17#note_261315
We need to be able to capture the file size of the VDS file which is the output of the segy-vds conversion.
...Linked issue: https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/issues/17#note_261315
We need to be able to capture the file size of the VDS file which is the output of the segy-vds conversion.
Otherwise there will be an additional call to SDMS service.
However, we'd like to find out if the response could be tailored to add more informationhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/220Add CRS database lookup to SEGYImport tool2023-11-16T12:21:10ZMorten OfstadAdd CRS database lookup to SEGYImport toolIt would be so much easier if it was possible to specify CRS by using the EPSG ID or UTM zone to look up the CRSWkt automatically, e.g. like this: --crs EPSG:23031 or --crs UTM-31N to get the WKT
```
PROJCS["ED50 / UTM zone 31N",GEOGCS["...It would be so much easier if it was possible to specify CRS by using the EPSG ID or UTM zone to look up the CRSWkt automatically, e.g. like this: --crs EPSG:23031 or --crs UTM-31N to get the WKT
```
PROJCS["ED50 / UTM zone 31N",GEOGCS["ED50",DATUM["European_Datum_1950",SPHEROID["International 1924",6378388,297],TOWGS84[-87,-98,-121,0,0,0,0]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4230"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",3],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","23031"]]
```
I looked up this at [https://epsg.io/23031](https://epsg.io/23031)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/204Add a mechanism to use an external application to get credentials2023-09-08T08:27:25ZMorten OfstadAdd a mechanism to use an external application to get credentialsIn order to get credentials that require a user to log in, it will be useful to run a separate executable. This is e.g. how git works (global config sets credential.helper to point to an executable, can be configured per URL prefix, see ...In order to get credentials that require a user to log in, it will be useful to run a separate executable. This is e.g. how git works (global config sets credential.helper to point to an executable, can be configured per URL prefix, see Git - gitcredentials Documentation (git-scm.com)). Integrating this directly in OpenVDS makes it easy for other applications to take advantage of.
The suggested implementation will add new global keys (valid for all cloud providers) credential_helper and credential_helper_args to the connection string format. If the credential_helper key is present, the executable pointed to will be run with the args from credential_helper_args and the URL as arguments, and the output will be parsed as a connection string and added to the remaining keys after removing the credential_helper and credential_helper_args keys. This allows other arguments like tolerance etc. to be passed on from the original connection string after using the credentials helper.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/191Getting histogram/statistics from OpenVDS data2023-06-22T14:11:50ZQiang FuGetting histogram/statistics from OpenVDS dataDoes OpenVDS data have embedded metadata for histogram/Statistics?
Go through all bricks to calculate the histogram or statistics could be quite expensive.Does OpenVDS data have embedded metadata for histogram/Statistics?
Go through all bricks to calculate the histogram or statistics could be quite expensive.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/190Add OSDU manifest generation capability2023-06-02T14:03:18ZMorten OfstadAdd OSDU manifest generation capabilityVDSInfo (or maybe a new vds utility) should be able to generate OSDU compliant JSON manifest for ingesting VDS. This was suggested by Juliana Fernandes (@fernandes_jfa) and would make life a lot easier when interacting with the OSDU inge...VDSInfo (or maybe a new vds utility) should be able to generate OSDU compliant JSON manifest for ingesting VDS. This was suggested by Juliana Fernandes (@fernandes_jfa) and would make life a lot easier when interacting with the OSDU ingestion service.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/187[question] SDMS v4 support2023-05-24T16:10:07ZFilip Brzęk[question] SDMS v4 supportHi,
might I ask, what's the support posture for SDMS v4 endpoints, that AFAIK are available for openvds from M17 [link to the swagger](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/sei...Hi,
might I ask, what's the support posture for SDMS v4 endpoints, that AFAIK are available for openvds from M17 [link to the swagger](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/v0.17.2/app/sdms-v4/docs/openapi.yaml?ref_type=tags)?
Regards,
Filiphttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/174Addition of valid input for SEGYImport2023-02-23T13:23:58ZAlexander JaustAddition of valid input for SEGYImportI wonder what would be the correct the conversion of SEG-Y files to VDS using `SEGYImport` when the provided options by `SEGYImport` are not rich enough.
Example: I have an attribute map in a SEG-Y file. However, none of the allowed na...I wonder what would be the correct the conversion of SEG-Y files to VDS using `SEGYImport` when the provided options by `SEGYImport` are not rich enough.
Example: I have an attribute map in a SEG-Y file. However, none of the allowed names for the `Attribute` property (`--attribute-name`: Amplitude (default), Attribute, Depth, Probability, Time, Vavg, Vint, or Vrms) are fitting my needs. The attribute name `Attribute` would be too vague for my use case and the other allowed names do not fit either.
- Should I import the SEG-Y file to VDS and afterwards change the name? I am not sure if that is possible, cf. #173.
- Could I provide a patch that extends SEGYImport by the units, names etc. that I would need?
- Should I fork and create my own SEGYImport? I would really like to avoid this.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/172KnownChannelNames class2023-02-23T12:36:51ZMorten OfstadKnownChannelNames classThere should be a KnownChannelNames class with the names from the documentation (https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/vds/specification/Metadata.html#named-channels) -- For C++ this is foun...There should be a KnownChannelNames class with the names from the documentation (https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/vds/specification/Metadata.html#named-channels) -- For C++ this is found in GlobalMetadataCommon.h, but it is not available in Python/Java.Morten OfstadMorten Ofstadhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/169SEGYImport documentation incomplete/inconsistent (legal values, default values)2023-01-23T12:28:35ZAlexander JaustSEGYImport documentation incomplete/inconsistent (legal values, default values)The comments here refer to the documentation of `SEGYImport` and the deep dive. I found some things that are inconsistent or easy to overlook depending on how one works with the tools provided by OpenVDS.
## CLI help
I find that docume...The comments here refer to the documentation of `SEGYImport` and the deep dive. I found some things that are inconsistent or easy to overlook depending on how one works with the tools provided by OpenVDS.
## CLI help
I find that documentation of `SEGYImport` in the terminal is lacking important information. I tend to use the terminal a lot and thus also use `SEGYImport --help` frequently.
The documentation of valid input values is a bit inconsistent. For some options (attribute name, attribute unit...) the output shows the accepted options.
```
...
--attribute-name <string>
The name of the primary VDS channel. The
name may be Amplitude (default), Attribute,
Depth, Probability, Time, Vavg, Vint, or
Vrms (default: Amplitude)
--attribute-unit <string>
The units of the primary VDS channel. The
unit name may be blank (default), ft, ft/s,
Hz, m, m/s, ms, or s
...
```
However, for "brick size" and "level of detail" levels the limits are not mentioned:
```
...
-b, --brick-size <value> The brick size for the volume data store.
--lod-levels <value> The number of LODs to generate.
...
```
When digging a bit deeper in the, [developer documentation](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/cppdoc/namespace/namespaceOpenVDS.html#_CPPv4N7OpenVDS26VolumeDataLayoutDescriptor9BrickSizeE) might even give the expectation that one should be allowed to use larger brick sizes that currently allowed. I assume that brick sizes >256 are only useful for 2D datasets. For level of detail levels, it is explained in the deep dive that the value can be at at least 0 and at most 12.
Additionally to that, neither does the [documentation](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/vds/deepdive/deepdive.html#brick-size) directly mention that the brick size has to be (certain) powers of two.
The output neither states the default value of the number of LODs. I expected that it would be zero as there is no general recommendation for number of levels in the deep dive (see also comment below).
The output of `SEGYImport --help` also deviates slightly from the [documentation on the homepage](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/tools/SEGYImport/README.html). I am not sure if it would be possible to synchonize the content of the README with the actual `SEGYImport` output.
There might be further options with incomplete documentation.
## Deep dive
### Margin
It is unclear to me what the correct margin size to chose is and what the actual default is without looking into the source code of `SEGYImport`. In the [deep dive](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/vds/deepdive/deepdive.html#margin-size) it is mentioned that a margin of 4 should be used as default and that the value is important when working with level of details. No connection to wavelets is mentioned.
In the [README](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/tools/SEGYImport/README.html) it mentions that the default value for the margin is 0 and 4 if one uses wavelet compression. Checking the source code confirms the statement of the README.
### Level of detail
It is explained when one should use level of details, but no general recommendation for how many level of detail levels one should generate is given (except if one wants to use FAST). However, `SEGYImport` chooses 2 as general default and 4 for poststack data.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/162Byte swap bug in Ibm2ieee2023-06-20T12:16:12ZArtem ВByte swap bug in Ibm2ieeeWe found that `Ibm2ieee` implementation does not take into account file and host endianness.
For example:
```
template<SEGY::Endianness ENDIANNESS, SEGY::BinaryHeader::DataSampleFormatCode FORMAT>
void copySamples(float * prTarget, con...We found that `Ibm2ieee` implementation does not take into account file and host endianness.
For example:
```
template<SEGY::Endianness ENDIANNESS, SEGY::BinaryHeader::DataSampleFormatCode FORMAT>
void copySamples(float * prTarget, const unsigned char * puSource, int iSampleMin, int iSampleMax)
{
if (ENDIANNESS == SEGY::Endianness::BigEndian)
{
nValue = (int)(puSource[iSample * 4 + 0] << 24 | puSource[iSample * 4 + 1] << 16 | puSource[iSample * 4 + 2] << 8 | puSource[iSample * 4 + 3]);
}
else
{
nValue = (int)(puSource[iSample * 4 + 3] << 24 | puSource[iSample * 4 + 2] << 16 | puSource[iSample * 4 + 1] << 8 | puSource[iSample * 4 + 0]);
}
}
```
use native conversion with real file endianess.
The original 'Ibm2ieee' form SEGY.cpp always swaps bytes:
```
template<SEGY::Endianness ENDIANNESS, SEGY::BinaryHeader::DataSampleFormatCode FORMAT>
void copySamples(float * prTarget, const unsigned char * puSource, int iSampleMin, int iSampleMax)
{
if (FORMAT == SEGY::BinaryHeader::DataSampleFormatCode::IBMFloat)
{
SEGY::Ibm2ieee(prTarget, puSource + iSampleMin * 4, iSampleMax - iSampleMin);
return;
}
}
void ibm2ieee(void * to, const void * from, size_t len)
{
...
#ifdef WIN32
fr = _byteswap_ulong(fr);
#else
fr = __builtin_bswap32(fr);
#endif // WIN32
...
}
```
It is needed to check real file and hosts endianess before swap bytes.
---
It makes wrong traces data for little-endian SEG-Ys with IBMFloats data formathttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/160Extend documentation of examples and supply sample files2022-11-18T12:56:34ZAlexander JaustExtend documentation of examples and supply sample filesIt is great that there are some examples on how to use OpenVDS. It would be even more helpful if there would be a short explanation of what each example does, what assumptions are made and, if necessary, that input files would be include...It is great that there are some examples on how to use OpenVDS. It would be even more helpful if there would be a short explanation of what each example does, what assumptions are made and, if necessary, that input files would be included. I think that would make life much easier for beginners.
For example, the [npz_to_vds.py](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/examples/NpzToVds/npz_to_vds.py) script is really helpful, but it would be nice if the input file would be provided with the script. It is not immediately clear that what assumptions on the file are made. When I looked into the file first I had the following questions:
- Why is there a `--npy` command line parameter that is never used?
- Why do the axis descriptors seem to expect x, y and z to be in a certain range [0,2000]?
- Why is the value range computed in the way it is computed? What is the "correct" way to give a value range? Should it be certain percentiles?
- Where is the input file or how can I create a valid input file myself?
- How does writing data via the page accessor actually work and where do I find more information about that?
- Do I have to use `open` and `close` for interaction with the VDS file or could I also use
```python
...
with openvds.create(
args.url,
args.connection,
layoutDescriptor,
axisDescriptors,
channelDescriptors,
metaData,
compressionMethod,
compressionTolerance
) as vds:
layout = openvds.getLayout(vds)
...
accessor.commit()https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/159Online Python documentation incomplete2022-11-21T09:34:46ZAlexander JaustOnline Python documentation incompleteIt seems that the online documentation of the Python interface is incomplete. I did not go through everything, but I found the following inconsistencies and problems.
Would it be possible to expose more documentation on the homepage? I ...It seems that the online documentation of the Python interface is incomplete. I did not go through everything, but I found the following inconsistencies and problems.
Would it be possible to expose more documentation on the homepage? I am not sure if some documentation is simply missing or not generated on purpose. In all cases that I have checked the classes and methods have a documentation accessible via Python's `help(...)` function.
## Observations
### VolumeDataLayoutDescriptor
I do not find anything about the constructors of `class VolumeDataLayoutDescriptor` in the [online documentation](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/python-api.html#openvds.VolumeDataLayoutDescriptor). When I query the documentation via Python I get much more information including the constructor.
That means if I do the following in a Python session
```text
import openvds
help(openvds.VolumeDataLayoutDescriptor)
```
gives the following (shortened) output
```text
Help on class VolumeDataLayoutDescriptor in module openvds.core:
class VolumeDataLayoutDescriptor(pybind11_builtins.pybind11_object)
| Method resolution order:
| VolumeDataLayoutDescriptor
| pybind11_builtins.pybind11_object
| builtins.object
|
| Methods defined here:
|
| __init__(...)
| __init__(*args, **kwargs)
| Overloaded function.
|
| 1. __init__(self: openvds.core.VolumeDataLayoutDescriptor) -> None
|
| 2. __init__(self: openvds.core.VolumeDataLayoutDescriptor, brickSize: OpenVDS::VolumeDataLayoutDescriptor::BrickSize, negativeMargin: int, positiveMargin: int, brickSize2DMultiplier: int, lodLevels: OpenVDS::VolumeDataLayoutDescriptor::LODLevels, options: OpenVDS::VolumeDataLayoutDescriptor::Options, fullResolutionDimension: int = 0) -> None
|
...
```
As you can see there is some information on the constructor.
### VolumeDataRequest
There are (at least?) two types of `VolumeDataRequest`, but only one is documented in the [online documentation](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/python-api.html#openvds.VolumeDataRequest).
The documentation explains the usage of an object of type `openvds.core.VolumeDataRequest`, see
```text
>>> import openvds
>>> data_request = openvds.VolumeDataRequest
>>> print(data_request)
<class 'openvds.core.VolumeDataRequest'>
>>> buffer = data_request.buffer
>>> data = data_request.data
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'openvds.core.VolumeDataRequest' has no attribute 'data'
```
This class does not have any attribute called `data` so the error is correct.
However, a function call to the function `requestVolumeSubset` of a `VolumeDataAccessManager` will return an object of type `openvds.volumedataaccess.VolumeDataRequest` with different interface which has a property called `data` instead of `buffer`.
```text
>>> import openvds
>>> data_request = openvds.volumedataaccess.VolumeDataRequest
>>> print(data_request)
<class 'openvds.volumedataaccess.VolumeDataRequest'>
>>> buffer = data_request.buffer
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'VolumeDataRequest' has no attribute 'buffer'
>>> data = data_request.data
```
This class does not have a `buffer` attribute so the error message is fine. It is not clear from the documentation of [`VolumeDataAccessManager`](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/python-api.html#id59) that the return type is not a `openvds.core.VolumeDataRequest`.
### openvds.IJKCoordinateTransformer
There is no documentation on the `IJKCoordinateTransformer` for Python even though it is accessible from Python
```text
>>> transformer = openvds.IJKCoordinateTransformer
>>> print(transformer)
<class 'openvds.core.IJKCoordinateTransformer'>
```
### VolumeDataAccessManager
The type annotation for the constructor of the `handle` seems to be off as it is `int`, but should be `openvds.core.VDS`.
```text
>>> vdam = openvds.VolumeDataAccessManager
>>> print(vdam)
<class 'openvds.volumedataaccess.VolumeDataAccessManager'>
>>> vdam = openvds.VolumeDataAccessManager(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/aej/software/openvds-3.0.3-install-python/python/openvds/volumedataaccess.py", line 184, in __init__
self._manager = openvds.core.getAccessManager(handle)
TypeError: getAccessManager(): incompatible function arguments. The following argument types are supported:
1. (handle: openvds.core.VDS) -> OpenVDS::VolumeDataAccessManager
Invoked with: 1
```
Update: In the last case, supplying an object of type `openvds.core.VDS` also fails so I assume the documentation might be off here or I misunderstand something.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/157Behavior for files with irregular inlines/crosslines2022-11-15T17:29:04ZAlena ChaikouskayaBehavior for files with irregular inlines/crosslines_(sorry, I will try to divide issue into parts, as for some reason it gets constantly marked as spam)_
While playing with openvds we accidentally created synthetic segy files which are imported into vds incorrectly (roundtrip breaks).
W..._(sorry, I will try to divide issue into parts, as for some reason it gets constantly marked as spam)_
While playing with openvds we accidentally created synthetic segy files which are imported into vds incorrectly (roundtrip breaks).
We are not sure how likely files like that can appear in reality, but our domain knowledge source tells us that it is possible in theory.
1. File [broken1.segy](/uploads/1e35d0885342952d3f3c5ec69273e02c/broken1.segy) with ilines `[1, 6, 11, 15]`
(Stride is 5, last element is at distance 4)
2. File [broken2.segy](/uploads/f2e69b64faf656cdb771ac2215264221/broken2.segy) with ilines `[1, 6, 11, 16, 21, 26, 27]`
(Stride is 5, last element is at distance 1)
Some rows just get lost, never to be found in vds.
Main observation is that as openvds for some reason on purpose ignores the distance between two last values, we need to supply a different distance to last element to reproduce this behavior.
How short/long is the distance of the last element might also matter as changing [this](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/tools/SEGYImport/SEGYImport.cpp#L2185) suspicious piece of code (as it seems that `a + (a-b)%c - b` is not divisible by `c`) fixed only one of those cases for me.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/153Cannot create IJKCoordinateTransformer for 2D dataset (Python)2022-10-31T14:31:13ZAlexander JaustCannot create IJKCoordinateTransformer for 2D dataset (Python)## Description
I am currently playing around with the creation of VDS files. I am especially interested in working via the Python interface and the different coordinate systems. I set up the a small [Python script](/uploads/8d79d553af1f...## Description
I am currently playing around with the creation of VDS files. I am especially interested in working via the Python interface and the different coordinate systems. I set up the a small [Python script](/uploads/8d79d553af1f908973720a1e03b21e06/write_2d_vds_data_testing.py) that creates a simple 2D dataset from a NumPy array with random content. Parts of the script is based on the `npz_to_vds.py` script from the examples. I would like to convert between inline/crossline coordinates, voxel coordinates and world coordinates.
In my script, the creation of the VDS file is successful. I also see that the file is recognized as 2D file by OpenVDS during writing since the chunks written to the page buffer are 4*brick_size. However, when I want to obtain the `IJKCoordinateTransformer` for this file, I run into the following exception
```text
Exception:
Dimension -1 is not a valid dimension. Dimensionality_Max is 6.
```
When I create a 3D file with only one coordinate in z direction, obtaining the transformer seems to be successful.
## Expectation
I obtain the coordinate transformer which allows me to transform between [different coordinate](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/cppdoc/struct/structOpenVDS_1_1IJKCoordinateTransformer.html) systems (ijk, inline/crossline etc.).
## Questions
- Is this behavior expected? I assumed that I could still work with the IJK transformer.
- Do I create the file as a 2D file in a wrong way?
- If the behavior is expected, would it be possible to extend the transformer to work with 2D data and/or as a quick fix to make the error message more expressive?
## System
- Arm64 MacOS 12.6
- VDS 3.0.3 with Python interfacehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/147Nearest interpolation does not choose the numerically nearest voxel2023-10-19T18:24:41ZErlend HårstadNearest interpolation does not choose the numerically nearest voxel
Hi,
Why does vds’ nearest interpolation interpolate in the range `[0, 1)` in around voxels, and not `[-0.5, 0.5)`? I find this a bit surprising. If I ask for voxel 1.99 in some dimension I would expect to get voxel 2 not 1, as 2 is nume...
Hi,
Why does vds’ nearest interpolation interpolate in the range `[0, 1)` in around voxels, and not `[-0.5, 0.5)`? I find this a bit surprising. If I ask for voxel 1.99 in some dimension I would expect to get voxel 2 not 1, as 2 is numerically “nearer” to 1.99 than 1https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/143No README.md for HueBDS2022-09-13T12:25:25ZMorten OfstadNo README.md for HueBDSThere is no README.md for the HueBDS tool and it doesn't get an entry in the documentation.There is no README.md for the HueBDS tool and it doesn't get an entry in the documentation.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/142Assert when trying to make slice on connection lost2022-08-30T08:30:45ZArtem ВAssert when trying to make slice on connection lostWe catch assertion in OpenVDS in base scenario with remote (S3) requests + lost internet conenction case:
P1. Open handle to remote S3 and make N requests:
```
std::vector<std::array<float, OpenVDS::Dimensionality_Max>> samplesPosition...We catch assertion in OpenVDS in base scenario with remote (S3) requests + lost internet conenction case:
P1. Open handle to remote S3 and make N requests:
```
std::vector<std::array<float, OpenVDS::Dimensionality_Max>> samplesPositions;
// Generate required points for small area
// And fetch via RequestVolumeSamples
auto vdsRequest(accessManager.RequestVolumeSamples(
OpenVDS::Dimensions_012,
0,
channelIndex,
reinterpret_cast<const float(*)[OpenVDS::Dimensionality_Max]>(samplesPositions.data()),
static_cast<int>(samplesPositions.size()),
OpenVDS::InterpolationMethod::Linear));
```
P2. Wrap P1 as std::future list:
```
std::vector<std::future<std::shared_ptr<VolumeDataRequestFloat>>> requests;
for (std::size_t t(0); t < 100; t++)
{
requests.emplace_back(std::async(makeRequest(t));
}
```
P3. Disconnect ethernet cable or turn off wifi connection
```
Expression: m_pendingDownloadRequests.find(chunk) == m_pendingDownloadRequests.end()
For information on how your program can cause an assertion
failure, see the Visual C++ documentation on asserts
(Press Retry to debug the application - JIT must be enabled)
---------------------------
Abort Retry Ignore
---------------------------
```
![image](/uploads/2b8169f2b0246eeadba6daaa02b6443d/image.png)
Looks like library should check chunk existage before access:
```
bool VolumeDataStoreIOManager::PrepareReadChunkImpl(const VolumeDataChunk &chunk, int adaptiveLevel, Error &error)
{
...
// HERE
if (m_pendingDownloadRequests.find(chunk) == m_pendingDownloadRequests.end())
return false
/// Continue
std::string url = CreateUrlForChunk(layerName, chunk.index);
auto transferHandler = std::make_shared<ReadChunkTransfer>(compressionInfo, (metadataManager != nullptr) ? parsedMetadata.CreateChunkMetadata() : std::vector<uint8_t>());
if (isConstantValue)
{
m_pendingDownloadRequests[chunk] = PendingDownloadRequest(transferHandler);
}
else
{
m_pendingDownloadRequests[chunk] = PendingDownloadRequest(m_ioManager->ReadObject(url, transferHandler, ioRange), transferHandler);
}
return true;
}
```https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/120Can't create VDS handle to read from S3 if remote VDS compressed with Wavelet...2022-06-03T11:07:05ZAlex TCan't create VDS handle to read from S3 if remote VDS compressed with WaveletNormalizeBlock methodHello!
I ran into a problem trying to open remote VDS on S3 compressed using `WaveletNormalizeBlock` method.
**The problem** description: I receive "illegal compression method" error trying to read from that VDS. It looks impossible to...Hello!
I ran into a problem trying to open remote VDS on S3 compressed using `WaveletNormalizeBlock` method.
**The problem** description: I receive "illegal compression method" error trying to read from that VDS. It looks impossible to read from `WaveletNormalizeBlock`-compressed VDS using VDSInfo or any other OpenVDS tool (e.g. slicedump sample), but I thought it is OK to use wavelet-compressed VDS just to read.
Moreover, such a VDS can't be read from even using OpenVDS+ library tools!
Surprisingly, if I use local file-based VDS with `WaveletNormalizeBlock` compression, all works perfectly.
If it's important, I have uploaded such a VDS to S3 using OpenVDS+ utils (SEGYImport/VDSCopy).
As I can see in OpenVDS sources, the reason is that ParseVDSJson.cpp module serializes `CompressionMethod::WaveletNormalizeBlock` enum to string as `"WaveletNormalizeBlock"` inside `std::string ToString(CompressionMethod compressionMethod)` function, but deserialize function `CompressionMethod CompressionMethodFromJson(Json::Value const &jsonCompressionMethod)` has the following code:
static CompressionMethod CompressionMethodFromJson(Json::Value const &jsonCompressionMethod) {
std::string compressionMethodString = jsonCompressionMethod.asString();
...
else if(compressionMethodString == "WaveletNormalizeBlockExperimental") // NOTE THIS
{
return CompressionMethod::WaveletNormalizeBlock;
}
...
else
{
throw Json::Exception("Illegal compression method");
}
}
So strings don't match each other (`"WaveletNormalizeBlock"` != `"WaveletNormalizeBlockExperimental"`) and we run into exception.
**My question** is: isn't this is a bug/typo in source code and can it be fixed? (for OpenVDS+ release too, if I may ask)
Or if this is intended behavior, can you please reveal the reason behind this logic?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/110Typo in volumedataaccess.py2022-05-04T07:15:08ZFilip BrzękTypo in volumedataaccess.pyPython's bidings for `requestVolumeSamples` have a typos, [line 522](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/python/openvds/volumedataaccess.py#L522) should have `samplePosit...Python's bidings for `requestVolumeSamples` have a typos, [line 522](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/python/openvds/volumedataaccess.py#L522) should have `samplePosition` positional argument passed, not `arr` that causes `NameError: name 'arr' is not defined`.
```Python
def requestVolumeSamples(self, samplePositions, data_out = None, dimensionsND = DimensionsND.Dimensions_012, lod = 0, channel = 0, interpolationMethod = InterpolationMethod.Cubic, replacementNoValue = None):
"""Request a set of samples from the volume. The samples are always in 32-bit floating point format.
Parameters
----------
samplePositions:
A set of voxel coordinates to obtain sample values for.
data_out : numpy.ndarray, optional
If specified, the data requested is copied to this array. Otherwise, a suitable numpy array is allocated.
dimensionsND : DimensionsND, optional
If specified, determine the dimensiongroup requested. Defaults to Dimensions_012
lod : int, optional
Which LOD level to request. Defaults to 0
channel : int, optional
Channel index. Defaults to 0.
interpolationMethod: InterpolationMethod, optional
Defaults to InterpolationMethod.Cubic
replacementNoValue: float, optional
If specified, NoValue data in the dataset is replaced with this value.
Returns
-------
request : VolumeDataRequest
An object encapsulating the request, the request state, and the requested data.
"""
if data_out is None:
data_out = self.allocateVolumeSamplesBuffer(len(samplePositions), channel)
else:
if data_out.nbytes < self.getVolumeSamplesBufferSize(sampleCount, channel):
raise ValueError("output array is too small to hold the requested data with format {}".format(str(VoxelFormat.Format_R32)))
req = VolumeDataRequest(
data_out = data_out,
dimensionsND = dimensionsND,
lod = lod,
channel = channel,
format = VoxelFormat.Format_R32,
replacementNoValue = replacementNoValue,
interpolationMethod = interpolationMethod)
req.samplePositions = _ndarraypositions(arr)
req._request = self._manager.requestVolumeSamples(
req.data_out,
req.dimensionsND,
req.lod,
req.channel,
req.samplePositions,
req.samplePositions.shape[0],
req.interpolationMethod,
req.replacementNoValue)
return req
```
PS. If you want I can create PR for that, together with smoke test with some comparison against `.getValue()`.
PSS. `sampleCount` in the branch where `data_out` is passed is also undefined.