Open VDS issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues2022-04-04T10:18:26Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/116ZLib version change2022-04-04T10:18:26ZPaal KvammeZLib version changeOpenZGY Version 2.3.2 no longer builds due to a 404 on http://zlib.net/zlib-1.2.11.tar.gz
The problem is that Zlib is hard coded at version 1.2.11 and downloaded as a source tarball. Which makes a lot of sense. The ZLib maintainers fixe...OpenZGY Version 2.3.2 no longer builds due to a 404 on http://zlib.net/zlib-1.2.11.tar.gz
The problem is that Zlib is hard coded at version 1.2.11 and downloaded as a source tarball. Which makes a lot of sense. The ZLib maintainers fixed a few bugs and bumped the version to 1.2.12. And then promptly removed 1.2.11 from the servers to make sure everybody uses the latest version. Thanks, guys.
From my point of view the issue is not critical since I can just patch the source before building.
FYI, I don't normally need to rebuild OpenVDS unless the version changes, but I was working on my build scripts.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/34zlib use intrinsics on windows2020-04-16T13:03:29ZJørgen Lindjorgen.lind@3lc.aizlib use intrinsics on windowsNeed to pick up the cpu architecture and set the correct cmake flagsNeed to pick up the cpu architecture and set the correct cmake flagsJørgen Lindjorgen.lind@3lc.aiJørgen Lindjorgen.lind@3lc.aihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/24ZGY import/export2024-03-07T09:18:54ZMorten OfstadZGY import/exportIt would be desirable to be able to import/export the ZGY format from Ocean/Petrel, this is dependent on the ZGY libraries being open-sourced.It would be desirable to be able to import/export the ZGY format from Ocean/Petrel, this is dependent on the ZGY libraries being open-sourced.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/145Wrong error message in OpenVDSInterfaceImpl::Close2022-09-26T14:42:45ZAnatoly YanchevskyWrong error message in OpenVDSInterfaceImpl::CloseIn function `OpenVDSInterfaceImpl::Close`:
`throw InvalidOperation(fmt::format("Flush in Close failed: {}", error.string).c_str());`
need to change on:
`throw InvalidOperation(fmt::format("Flush in Close failed: {}", flashError.string...In function `OpenVDSInterfaceImpl::Close`:
`throw InvalidOperation(fmt::format("Flush in Close failed: {}", error.string).c_str());`
need to change on:
`throw InvalidOperation(fmt::format("Flush in Close failed: {}", flashError.string).c_str());`https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/161World coordinates when cdp is not defined2022-11-30T13:39:02ZAlena ChaikouskayaWorld coordinates when cdp is not definedWe believe that this case is unlikely to happen in real files, but wanted to point that out anyway.
When file has no cdp information, asking for information in World coordinates returns data in Annotation coordinate system. If transfor...We believe that this case is unlikely to happen in real files, but wanted to point that out anyway.
When file has no cdp information, asking for information in World coordinates returns data in Annotation coordinate system. If transformation with missing World coordinates is impossible, I would have expected some error message along the way.
SEGY [without_cdp.segy](/uploads/2af802cd954648a22c6a8b03c6241f54/without_cdp.segy):
```
spec.samples = [4, 8]
spec.ilines = [3, 4, 5]
spec.xlines = [10, 11]
DelayRecordingTime: 5,
```
openvds:
```
auto transformer = OpenVDS::IJKCoordinateTransformer(layout);
auto annotation = transformer.IJKIndexToAnnotation({0, 0, 0})
auto world = transformer.IJKIndexToWorld({ 0, 0, 0 });
```
Result:
```
Annotation: 3 10 5
World: 3 10 -5
```
World coordinates created from transformed I/J are same as annotation, which is misleading.
World coordinate created from transformed K is `-Time` (also in files with cdp), which seems strange, though somewhat understandable.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/136When I execute VdsInfo on sd://<>//<>/<> using python sub process, getting t...2022-09-05T14:48:30Zsangamesh hooliWhen I execute VdsInfo on sd://<>//<>/<> using python sub process, getting truncated response backhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/133When I am doing Vdscopy from local windows machine using python script using ...2022-08-25T04:10:53Zsangamesh hooliWhen I am doing Vdscopy from local windows machine using python script using sub process getting'[Dimensions_012LOD0/50: sdapi 3.14.0 - : Encountered network error when sending http request]\nVolumeDataAccessManager destructor: there where upload errors\n''[Dimensions_012LOD0/50: sdapi 3.14.0 - : Encountered network error when sending http request]\nVolumeDataAccessManager destructor: there where upload errors\n'https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/196Wavelet compression in Python2023-08-29T14:41:08ZVasilii SinkevichWavelet compression in PythonHi,
I am investigating VDS capabilities and trying to convert numpy array into VDS using Wavelet compression.
I am using the example provided along the source code (npy_to_vds.py) with minimal changes
Everything works fine with no com...Hi,
I am investigating VDS capabilities and trying to convert numpy array into VDS using Wavelet compression.
I am using the example provided along the source code (npy_to_vds.py) with minimal changes
Everything works fine with no compression or with Zip or RLE compression, I can write and read data back, but if I try using Wavelet, like this:
```python
compressionMethod = openvds.CompressionMethod(1)
```
the code execution gets stuck at committing changes
```python
accessor.commit()
```
regardless of compression tolerance value. It just hangs here for ages.
I am wondering if python port is actual OpenVDS+ binding, maybe it is actually OpenVDS (without plus) that do not support wavelet compression?
If it *is* OpenVDS+, could you please provide a sample of the code (or maybe edit npy_to_vds.py) that saves VDS format with wavelet compression?
EDIT: I am working on Debian 9, python 3.9, openvds 3.2.6
Thanks,
Vasiliihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/222VoumeData class not included in Java Library2024-02-26T20:07:38ZJulien LacosteVoumeData class not included in Java LibraryThe VolumeData class is not generated in the Java version.
Apparently it's missing from de the CmakeLists.txtThe VolumeData class is not generated in the Java version.
Apparently it's missing from de the CmakeLists.txthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/3VolumeDataAccessor Clone() method is not implemented2020-01-03T14:48:54ZMorten OfstadVolumeDataAccessor Clone() method is not implementedThis method is used to clone a VolumeDataAccessor e.g. when used in a firstprivate() clause of an OpenMP parallel construct. It implements ref-counted shared access to the underlying VolumeDataPageAccessor while maintaining the pinned pa...This method is used to clone a VolumeDataAccessor e.g. when used in a firstprivate() clause of an OpenMP parallel construct. It implements ref-counted shared access to the underlying VolumeDataPageAccessor while maintaining the pinned page state separately for each thread. A test needs to be added to ensure that this works as intended.Version 1.0Morten OfstadMorten Ofstadhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/176VDS with LOD levels unexpectedly large2023-03-22T14:43:59ZAlexander JaustVDS with LOD levels unexpectedly largeI find VDS data sets to increase much more in size when LOD levels are added than expected.
## Obervation/Experiment
I created a VDS data sets from a volve data set (`ST0202R08_PS_PSDM_FULL_OFFSET_PP_TIME.MIG_FIN.POST_STACK.3D.JS-01753...I find VDS data sets to increase much more in size when LOD levels are added than expected.
## Obervation/Experiment
I created a VDS data sets from a volve data set (`ST0202R08_PS_PSDM_FULL_OFFSET_PP_TIME.MIG_FIN.POST_STACK.3D.JS-017534.segy`, about 1GiB) for different LOD levels. This is a 3D post-stack dataset such that I would expect an increase in file size of about 15% if all LOD levels are created. The major increase should appear for the first few LOD levels.
I observe the following file sizes:
| LOD level | Size in MiB | Relative size |
|-----------|-------------|---------------|
| 0 | 864,04 | 100% |
| 1 | 1248,06 | 144% |
| 2 | 1456,09 | 169% |
| 3 | 1456,09 | 169% |
| 4 | 1454,31 | 168% |
| 5 | 1456,09 | 169% |
| 6 | 1456,09 | 169% |
| 7 | 1456,09 | 169% |
| 8 | 1456,09 | 169% |
| 9 | 1454,88 | 168% |
| 10 | 1456,10 | 169% |
| 11 | 1456,10 | 169% |
| 12 | 1456,11 | 169% |
Now I see that the file size increases by more than 50%. If I scale my estimate I see that the increase at least flattens out quickly what agrees with the geometric series. However, I also see a small dip for LOD level 10 and 11 which I don't exactly understand, but maye that is due to some collapsed blocks.
I used the default settings, but also tested with LOD level and compression set explicitly with the same results for the file size. The file is stored on a local hard drive and file size is checked with `du`.
I also tested with a larger SEGY where VDS with no LOD levels is 11 GiB large. When adding 4 LOD levels the file size goes up to 20 GiB. This is an increase of file size by even more than **80%**.
## Questions
Is this increase in file size expected? I assumed that the file size increase should be according to the [geometric series](https://en.wikipedia.org/wiki/Geometric_series) with `a=1` and `r=(1/2)^d` which would be `r=1/8` for 3D data sets.
## Platform
* Apple Arm M1 Max
* MacOS 13.2.1
* OpenVDS 3.0.3 (compiled from source)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/99VDS roundtrip to dms fails.2021-12-20T09:32:27ZPaal KvammeVDS roundtrip to dms fails.OpenVDS #9d3e767e
I did the following (partly redacted)
- VDSCopy Volve_from_segy.vds sd://tenant/project/tmp.vds
- VDSCopy sd://tenant/project/tmp.vds roundtrip.vds
The second line resulted in an error about "TraceDimensions_012LOD0/C...OpenVDS #9d3e767e
I did the following (partly redacted)
- VDSCopy Volve_from_segy.vds sd://tenant/project/tmp.vds
- VDSCopy sd://tenant/project/tmp.vds roundtrip.vds
The second line resulted in an error about "TraceDimensions_012LOD0/ChunkMetadata/0" not being found.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/230VDSInfo to point to VDS folder in SD-STORE2024-02-27T07:20:04ZDebasis ChatterjeeVDSInfo to point to VDS folder in SD-STORE@Ofstad - Please check the following.
The VDS folder has been created successfully from SegYImport step (converter).
We can check stat from sdutil.
We can also see the folder content from Backend (Amazon S3).
This is performed using AWS...@Ofstad - Please check the following.
The VDS folder has been created successfully from SegYImport step (converter).
We can check stat from sdutil.
We can also see the folder content from Backend (Amazon S3).
This is performed using AWS/M22/Preship.
Always the same error.
[Option 'c' does not exist]
VDSInfo –connection “sdauthorityurl=https://prsh.testing.preshiptesting.osdu.aws/api/seismic-store/v3;sdapikey=;sdtoken=access token" sd://osdu/subproject/debasi.0a2ed74b-c8c9-4289-a4b0-ce8e135e2cbd.vds
VDSInfo –connection “sdauthorityurl=https://prsh.testing.preshiptesting.osdu.aws/api/seismic-store/v3;sdtoken=access token" sd://osdu/subproject/debasi.0a2ed74b-c8c9-4289-a4b0-ce8e135e2cbd.vds
VDSInfo –connection “sdauthorityurl=https://prsh.testing.preshiptesting.osdu.aws/api/seismic-store/v3;sdapikey=ABC;sdtoken=access token" sd://osdu/subproject/debasi.0a2ed74b-c8c9-4289-a4b0-ce8e135e2cbd.vdshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/231VDSInfo showing textual header for a VDS file that has been converted from SegY2024-02-27T07:06:58ZDebasis ChatterjeeVDSInfo showing textual header for a VDS file that has been converted from SegYVDSInfo --metadata-name TextHeader -e -w 80 <data set>
Is there an option to show actual text content?
Currently when I try the following command, it does not decode to text.
```
C:\openvds-3.0.4\bin\msvc_140>VDSInfo --metadata-name ...VDSInfo --metadata-name TextHeader -e -w 80 <data set>
Is there an option to show actual text content?
Currently when I try the following command, it does not decode to text.
```
C:\openvds-3.0.4\bin\msvc_140>VDSInfo --metadata-name TextHeader -e -w 80 debasis9Feb.vds
{
"category" : "SEGY",
"name" : "TextHeader",
"type" : "BLOB",
"value" : "w0DxQM....AQEA="
}
C:\openvds-3.0.4\bin\msvc_140>
```
cc @Ofstadhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/57[VDSInfo] Query all metadata with single command2020-07-06T09:02:52ZJesse Hudgens[VDSInfo] Query all metadata with single commandThere is interest in querying all available metadata using a single command in VDSInfo rather specifying a name or category predicate.There is interest in querying all available metadata using a single command in VDSInfo rather specifying a name or category predicate.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/138VDSError: [Chunk: 0, Channel: 1 LOD: 0: Trying to read from a layer that has ...2022-08-25T04:10:22Zsangamesh hooliVDSError: [Chunk: 0, Channel: 1 LOD: 0: Trying to read from a layer that has not been added] VolumeDataAccessManager destructor: there where upload errorsWhen I do VDS copy from local machine to sd://<>/<>/<> getting below error
VDSError: [Chunk: 0, Channel: 1 LOD: 0: Trying to read from a layer that has not been added] VolumeDataAccessManager destructor: there where upload errorsWhen I do VDS copy from local machine to sd://<>/<>/<> getting below error
VDSError: [Chunk: 0, Channel: 1 LOD: 0: Trying to read from a layer that has not been added] VolumeDataAccessManager destructor: there where upload errorshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/212VDSCopy hanging when uploading to Seismic DDMS2023-10-23T11:39:11Zvinicius Vicente Silva RosaVDSCopy hanging when uploading to Seismic DDMSI am attempting to upload a local VDS file (1.5TB) to a SD Path, and after approximately an hour, there is no visible progress in the file upload, creating the impression that the process is stalled. No error messages are being displayed...I am attempting to upload a local VDS file (1.5TB) to a SD Path, and after approximately an hour, there is no visible progress in the file upload, creating the impression that the process is stalled. No error messages are being displayed. I suspect it may be related to the token refresh.
We are using the command line bellow:
OSDU/ADME M16
lIB: VDSCopy - OpenVDS+ 3.3.0 installed on Linux
```bash
VDSCopy -a 01 -a 02 -a 12 --tolerance=1.0 --compression-method=Wavelet -d 'sdAuthorityUrl=
https://{HOST}.energy.azure.com/seistore-svc/api/v3;authTokenUrl=https://login.microsoftonline.com/{TENANT}/oauth2/v2.0/token/;client_id={APP_ID};client_secret={APP_SECRET};scopes={APP_ID}/.default;'
'/local_disk0/vds/FILE.vds' 'sd://{TENANT}/{SUBPROJECT}/dataset_name.vds'
```
The intention of the command above is to authenticate using ClientID and ClientSecrect.
The upload completes successfully when the file is processed within an hour or less.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/1vanilla checkout build fails on linux2019-12-20T08:47:12ZJackie Livanilla checkout build fails on linuxfails with:
```
open-vds/src/IO/IOManagerAzure.cpp:20:10: fatal error: was/common.h: No such file or directory
#include <was/common.h>
^~~~~~~~~~~~~~
compilation terminated.
src/CMakeFiles/openvds_objects.dir/build.make:127: ...fails with:
```
open-vds/src/IO/IOManagerAzure.cpp:20:10: fatal error: was/common.h: No such file or directory
#include <was/common.h>
^~~~~~~~~~~~~~
compilation terminated.
src/CMakeFiles/openvds_objects.dir/build.make:127: recipe for target 'src/CMakeFiles/openvds_objects.dir/IO/IOManagerAzure.cpp.o' failed
```https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/10UploadRequestAWS gets destructed while FlushUploadQueue is waiting for it to ...2019-12-20T08:47:00ZMorten OfstadUploadRequestAWS gets destructed while FlushUploadQueue is waiting for it to finishWe are seeing 2 types of mutex error appear randomly, and abort the process: unlocking an unowned mutex, and mutex destroyed whilst busy.
The stack for unlocking unowned mutex is:
```
ucrtbased.dll!issue_debug_notification(const wc...We are seeing 2 types of mutex error appear randomly, and abort the process: unlocking an unowned mutex, and mutex destroyed whilst busy.
The stack for unlocking unowned mutex is:
```
ucrtbased.dll!issue_debug_notification(const wchar_t * const message) Line 28 C++
ucrtbased.dll!__acrt_report_runtime_error(const wchar_t * message) Line 154 C++
ucrtbased.dll!abort() Line 61 C++
msvcp140d.dll!_Thrd_abort(const char * msg) Line 18 C++
msvcp140d.dll!_Mtx_unlock(_Mtx_internal_imp_t * mtx) Line 167 C++
> otter.dll!std::_Mutex_base::unlock() Line 67 C++
otter.dll!std::unique_lock<std::mutex>::~unique_lock<std::mutex>() Line 188 C++
otter.dll!OpenVDS::UploadRequestAWS::WaitForFinish() Line 302 C++
otter.dll!OpenVDS::VolumeDataAccessManagerImpl::FlushUploadQueue() Line 598 C++
otter.dll!OpenVDS::VolumeDataPageAccessorImpl::Commit() Line 504 C++
otter.dll!otter_core::VdsImage::writeToBlock(int block_index, void * buffer, unsigned __int64 buffer_size) Line 410 C++
```
The stack for the mutex destroyed whilst busy is:
```
ucrtbased.dll!issue_debug_notification(const wchar_t * const message) Line 28 C++
ucrtbased.dll!__acrt_report_runtime_error(const wchar_t * message) Line 154 C++
ucrtbased.dll!abort() Line 61 C++
msvcp140d.dll!_Thrd_abort(const char * msg) Line 18 C++
msvcp140d.dll!_Mtx_destroy_in_situ(_Mtx_internal_imp_t * mtx) Line 65 C++
> otter.dll!std::_Mutex_base::~_Mutex_base() Line 44 C++
otter.dll!std::mutex::~mutex() C++
otter.dll!OpenVDS::UploadRequestAWS::~UploadRequestAWS() C++
otter.dll!OpenVDS::UploadRequestAWS::`scalar deleting destructor'(unsigned int) C++
otter.dll!std::_Destroy_in_place<OpenVDS::UploadRequestAWS>(OpenVDS::UploadRequestAWS & _Obj) Line 242 C++
otter.dll!std::_Ref_count_obj2<OpenVDS::UploadRequestAWS>::_Destroy() Line 1504 C++
otter.dll!std::_Ref_count_base::_Decref() Line 651 C++
otter.dll!std::_Ptr_base<OpenVDS::UploadRequestAWS>::_Decref() Line 882 C++
otter.dll!std::shared_ptr<OpenVDS::UploadRequestAWS>::~shared_ptr<OpenVDS::UploadRequestAWS>() Line 1132 C++
otter.dll!OpenVDS::upload_callback(const Aws::S3::S3Client * client, const Aws::S3::Model::PutObjectRequest & putRequest, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & outcome, std::weak_ptr<OpenVDS::UploadRequestAWS> weak_upload) Line 228 C++
otter.dll!OpenVDS::UploadRequestAWS::run::__l2::<lambda>(const Aws::S3::S3Client * client, const Aws::S3::Model::PutObjectRequest & putRequest, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & outcome, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & __formal) Line 293 C++
otter.dll!std::_Invoker_functor::_Call<void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &) &,Aws::S3::S3Client const *,Aws::S3::Model::PutObjectRequest const &,Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> const &,std::shared_ptr<Aws::Client::AsyncCallerContext const> const &>(OpenVDS::UploadRequestAWS::run::__l2::void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &) & _Obj, const Aws::S3::S3Client * && <_Args_0>, const Aws::S3::Model::PutObjectRequest & <_Args_1>, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & <_Args_2>, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & <_Args_3>) Line 1579 C++
otter.dll!std::invoke<void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &) &,Aws::S3::S3Client const *,Aws::S3::Model::PutObjectRequest const &,Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> const &,std::shared_ptr<Aws::Client::AsyncCallerContext const> const &>(OpenVDS::UploadRequestAWS::run::__l2::void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &) & _Obj, const Aws::S3::S3Client * && <_Args_0>, const Aws::S3::Model::PutObjectRequest & <_Args_1>, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & <_Args_2>, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & <_Args_3>) Line 1579 C++
otter.dll!std::_Invoker_ret<void,1>::_Call<void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &) &,Aws::S3::S3Client const *,Aws::S3::Model::PutObjectRequest const &,Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> const &,std::shared_ptr<Aws::Client::AsyncCallerContext const> const &>(OpenVDS::UploadRequestAWS::run::__l2::void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &) & <_Vals_0>, const Aws::S3::S3Client * && <_Vals_1>, const Aws::S3::Model::PutObjectRequest & <_Vals_2>, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & <_Vals_3>, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & <_Vals_4>) Line 1598 C++
otter.dll!std::_Func_impl_no_alloc<void <lambda>(const Aws::S3::S3Client *, const Aws::S3::Model::PutObjectRequest &, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> &, const std::shared_ptr<Aws::Client::AsyncCallerContext const> &),void,Aws::S3::S3Client const *,Aws::S3::Model::PutObjectRequest const &,Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> const &,std::shared_ptr<Aws::Client::AsyncCallerContext const> const &>::_Do_call(const Aws::S3::S3Client * && <_Args_0>, const Aws::S3::Model::PutObjectRequest & <_Args_1>, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & <_Args_2>, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & <_Args_3>) Line 927 C++
aws-cpp-sdk-s3.dll!std::_Func_class<void,Aws::S3::S3Client const *,Aws::S3::Model::PutObjectRequest const &,Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> const &,std::shared_ptr<Aws::Client::AsyncCallerContext const> const &>::operator()(const Aws::S3::S3Client * <_Args_0>, const Aws::S3::Model::PutObjectRequest & <_Args_1>, const Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> & <_Args_2>, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & <_Args_3>) Line 970 C++
aws-cpp-sdk-s3.dll!Aws::S3::S3Client::PutObjectAsyncHelper(const Aws::S3::Model::PutObjectRequest & request, const std::function<void __cdecl(Aws::S3::S3Client const *,Aws::S3::Model::PutObjectRequest const &,Aws::Utils::Outcome<Aws::S3::Model::PutObjectResult,Aws::Client::AWSError<enum Aws::S3::S3Errors>> const &,std::shared_ptr<Aws::Client::AsyncCallerContext const> const &)> & handler, const std::shared_ptr<Aws::Client::AsyncCallerContext const> & context) Line 3316 C++
aws-cpp-sdk-s3.dll!Aws::S3::S3Client::PutObjectAsync::__l2::<lambda>() Line 3311 C++
aws-cpp-sdk-s3.dll!std::_Invoker_functor::_Call<void <lambda>(void) &>(Aws::S3::S3Client::PutObjectAsync::__l2::void <lambda>(void) & _Obj) Line 1579 C++
aws-cpp-sdk-s3.dll!std::invoke<void <lambda>(void) &>(Aws::S3::S3Client::PutObjectAsync::__l2::void <lambda>(void) & _Obj) Line 1579 C++
aws-cpp-sdk-s3.dll!std::_Invoker_ret<std::_Unforced,0>::_Call<void <lambda>(void) &>(Aws::S3::S3Client::PutObjectAsync::__l2::void <lambda>(void) & <_Vals_0>) Line 1615 C++
aws-cpp-sdk-s3.dll!std::_Call_binder<std::_Unforced,void <lambda>(void),std::tuple<>,std::tuple<>>(std::_Invoker_ret<std::_Unforced,0> __formal, std::integer_sequence<unsigned __int64> __formal, Aws::S3::S3Client::PutObjectAsync::__l2::void <lambda>(void) & _Obj, std::tuple<> & _Tpl, std::tuple<> && _Ut) Line 1402 C++
aws-cpp-sdk-s3.dll!std::_Binder<std::_Unforced,void <lambda>(void)>::operator()<>() Line 1442 C++
aws-cpp-sdk-s3.dll!std::_Invoker_functor::_Call<std::_Binder<std::_Unforced,void <lambda>(void)> &>(std::_Binder<std::_Unforced,void <lambda>(void)> & _Obj) Line 1579 C++
aws-cpp-sdk-s3.dll!std::invoke<std::_Binder<std::_Unforced,void <lambda>(void)> &>(std::_Binder<std::_Unforced,void <lambda>(void)> & _Obj) Line 1579 C++
aws-cpp-sdk-s3.dll!std::_Invoker_ret<void,1>::_Call<std::_Binder<std::_Unforced,void <lambda>(void)> &>(std::_Binder<std::_Unforced,void <lambda>(void)> & <_Vals_0>) Line 1598 C++
aws-cpp-sdk-s3.dll!std::_Func_impl_no_alloc<std::_Binder<std::_Unforced,void <lambda>(void)>,void>::_Do_call() Line 927 C++
aws-cpp-sdk-core.dll!std::_Func_class<void>::operator()() Line 970 C++
aws-cpp-sdk-core.dll!Aws::Utils::Threading::DefaultExecutor::SubmitToThread::__l2::<lambda>() Line 29 C++
aws-cpp-sdk-core.dll!std::_Invoker_functor::_Call<void <lambda>(void)>(Aws::Utils::Threading::DefaultExecutor::SubmitToThread::__l2::void <lambda>(void) && _Obj) Line 1579 C++
aws-cpp-sdk-core.dll!std::invoke<void <lambda>(void)>(Aws::Utils::Threading::DefaultExecutor::SubmitToThread::__l2::void <lambda>(void) && _Obj) Line 1579 C++
aws-cpp-sdk-core.dll!std::thread::_Invoke<std::tuple<void <lambda>(void)>,0>(void * _RawVals) Line 43 C++
ucrtbased.dll!thread_start<unsigned int (__cdecl*)(void *),1>(void * const parameter) Line 97 C++
```Jørgen Lindjorgen.lind@3lc.aiJørgen Lindjorgen.lind@3lc.aihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/173Updating information stored in a VDS (Are VDS datasets mutable?)2023-03-27T06:48:15ZAlexander JaustUpdating information stored in a VDS (Are VDS datasets mutable?)I was playing around with OpenVDS to figure out whether and to what extent VDS datasets are mutable or not. My big question is: What parts of a VDS dataset are mutable. If there are any parts that are mutable, what is the correct way to ...I was playing around with OpenVDS to figure out whether and to what extent VDS datasets are mutable or not. My big question is: What parts of a VDS dataset are mutable. If there are any parts that are mutable, what is the correct way to change these parts.
Either way it has different implications for certain workflows (to me). This concerns metadata as well as channel data stored within the VDS.
1. If a VDS dataset is always immutable, I can be sure that nobody with accidentally change/break a VDS dataset.
2. If a VDS dataset is mutable, I could update some fields if, e.g., `SEGYImport` does not accept certain names/units during ingestion and/or update data within the VDS, e.g. add fast slice, an additional channel or update data within a channel without recreating the VDS dataset from scratch.
I potentially do some "stupid" things here, but I also try to model a worst case scenario here like "what is the worst thing somebody can do wrong?".
I found an [older issue](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/28) which mentions the addition of LOD levels to a VDS, but it does not specify whether this would happen in-place or would create a new VDS dataset.
## Observations / Experiments
1. I am able to add additional channel data to an existing channel of a VDS dataset. The additional data seems to hide the initially available channel data.
For testing I created a small Python script: [create_and_change_vds_inplace_small.py](/uploads/72d056826ce7bf455f1b69ddf023fec2/create_and_change_vds_inplace_small.py). The script creates a small artifical VDS dataset with one channel. The script carries out the following steps.
- Create VDS dataset with all values in the channel are set to `1`.
- Close file hande such that it can be written to disk.
- Open the file and extract a slice, copy the slice data and close the file.
- Open the file and get an AccessManager, write the value `2*old_value` (`2` in this case) to the VDS dataset and close the file.
- Open the file and extract a slice, copy the slice data and close the file.
- Plot the slice data.
When I run the script that the data extracted from the VDS indeed changes. For the first slice I get constant `1` values and constant `2` values for the second time I extract a slice. I am not sure if one can still access the "old" data. The file size increases so it appears to me as if the old data is still stored in the VDS dataset.
Is this behavior intended? If so, how can I access the "old" data? Would it be possible to actually update the data without increasing the file size?
2. I tried to update the metadata. For that I wrote a [small C++ code](/uploads/013acdfdfe460bef0d5dbec4b517ee98/update_metadata.cpp) as C++ seemed to have (more?) direct access to the `MetadataWriteAccess` object than Python. The code basically opens a specified VDS dataset and replaces the `ImportTimeStamp` values with some unrealistic time stamp.
The code executes, but gives a segmentation fault (`Segmentation fault: 11`). From debugging I concluded that the segmentation fault rises when the VDS dataset is being closed. Calling the `SetMetadataString` function seems to be fine.
Is this behavior intended? I guess the segmentation fault should never happen, but to me it is not clear if that is an error when updating the VDS dataset or it is a side effect of illegally writing the metadata.
## Platform
* Apple Arm M1 Max
* MacOS 13.1
* OpenVDS 3.0.3 (compiled from source)