Open VDS issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues2023-10-11T15:13:35Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/214Job Failed #2191076. Openvds-ingestion image for tag 3.3.0 not created2023-10-11T15:13:35ZAndrei Skorkin [EPAM / GCP]Job Failed #2191076. Openvds-ingestion image for tag 3.3.0 not createdJob [#2191076](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/jobs/2191076) failed for 50c3f90f370390cfd5d23defeb7603bbfa01374c
During the release cycle, we automatically change the latest ima...Job [#2191076](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/jobs/2191076) failed for 50c3f90f370390cfd5d23defeb7603bbfa01374c
During the release cycle, we automatically change the latest image tag to the latest one presented here for **openvds-ingestion** image. In this case it was **3.3.0** ( https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-gcp-provisioning/-/blob/release/0.23/helm/osdu-infra-baremetal/values.yaml?ref_type=heads#L182 ). But due to the failure of the mentioned job, image does not exist and we need to manually fix the wrong version in the values.yaml file to make a deployment.
Could you please check this?
How to overcome similar problems in the future?
Thanks
CC: @Yauhen_Shaliou @Yan_SushchynskiMorten OfstadMorten Ofstadhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/213NEWBIE - installation on ARM 64 Linux graviton2024-02-01T07:16:11ZKlaas KosterNEWBIE - installation on ARM 64 Linux gravitonFollowing the instructions, I executed:
1) cmake ..
2) make -j8
3) make install
No errors or warnings are generated, and five executables are placed in Dist/OpenVDS/bin.
Two issues:
1) The README file states that ./SEGYImport should sho...Following the instructions, I executed:
1) cmake ..
2) make -j8
3) make install
No errors or warnings are generated, and five executables are placed in Dist/OpenVDS/bin.
Two issues:
1) The README file states that ./SEGYImport should show the Wavelet Compression option, but it does not.
2) I cannot find the .whl file anywhere that would allow me to use 'pip install' to get Python to work with OpenVDS.
What did I do wrong or forget?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/212VDSCopy hanging when uploading to Seismic DDMS2023-10-23T11:39:11Zvinicius Vicente Silva RosaVDSCopy hanging when uploading to Seismic DDMSI am attempting to upload a local VDS file (1.5TB) to a SD Path, and after approximately an hour, there is no visible progress in the file upload, creating the impression that the process is stalled. No error messages are being displayed...I am attempting to upload a local VDS file (1.5TB) to a SD Path, and after approximately an hour, there is no visible progress in the file upload, creating the impression that the process is stalled. No error messages are being displayed. I suspect it may be related to the token refresh.
We are using the command line bellow:
OSDU/ADME M16
lIB: VDSCopy - OpenVDS+ 3.3.0 installed on Linux
```bash
VDSCopy -a 01 -a 02 -a 12 --tolerance=1.0 --compression-method=Wavelet -d 'sdAuthorityUrl=
https://{HOST}.energy.azure.com/seistore-svc/api/v3;authTokenUrl=https://login.microsoftonline.com/{TENANT}/oauth2/v2.0/token/;client_id={APP_ID};client_secret={APP_SECRET};scopes={APP_ID}/.default;'
'/local_disk0/vds/FILE.vds' 'sd://{TENANT}/{SUBPROJECT}/dataset_name.vds'
```
The intention of the command above is to authenticate using ClientID and ClientSecrect.
The upload completes successfully when the file is processed within an hour or less.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/211Any advices for saving vds file?2023-09-29T09:15:42Znanting liuAny advices for saving vds file?- SEGYImport [OPTION...] <input file>
- use _–url <string>_ saving vds file to cloud environment, then vds file will be splited into several files(Dimensions_012LOD0、VolumeDataLayout...etc.)
- use _–vdsfile <string>_ saving vds file to l...- SEGYImport [OPTION...] <input file>
- use _–url <string>_ saving vds file to cloud environment, then vds file will be splited into several files(Dimensions_012LOD0、VolumeDataLayout...etc.)
- use _–vdsfile <string>_ saving vds file to local file system,then vds file will be completed with a file ending in .vds(test.vds)
- when i requst data from a file ending in .vds it will be so fast.
- And any difference from -url to -vdsfile ? its all use to set path of output VDS file. I would like to know how to use these two parameters correctly to improve the efficiency in querying data.
- thank u so much.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/210Error uploading VDS into SD Path using OpenVDS+2023-11-14T22:11:13ZJuliana Fernandesjuliana.fernandes@iesbrazil.com.brError uploading VDS into SD Path using OpenVDS+Hello,
I`m trying to upload a local VDS into a SD Path in AWS M20 pre-shipping and get an SDMS error (wrong location).
Some values where provided by e-mail, so I will not paste here, but if you need to test just let me know.
The command...Hello,
I`m trying to upload a local VDS into a SD Path in AWS M20 pre-shipping and get an SDMS error (wrong location).
Some values where provided by e-mail, so I will not paste here, but if you need to test just let me know.
The command I'm using is:
```
VDSCopy.exe -d "SdAuthorityUrl=https://prsh.testing.preshiptesting.osdu.aws/api/seismic-store/v3;SdApiKey=ABC;AuthTokenUrl={{received_by_email}};client_id={{received_by_email}};client_secret={{received_by_email}};grant_type=refresh_token;refresh_token={{generated_in_the_login}};LegalTag=osdu-public-usa-dataset-1;scopes=openid email;Region=us-east-2" E:\Juliana\osdu\osdu_test\ST0202R08_PS_PSDM_RAW_PP_TIME_MIG_RAW_POST_STACK_3D_JS_017534_tol1_JFA.vds sd://osdu/vdstestsjfa/ST0202R08_PS_PSDM_RAW_PP_TIME_MIG_RAW_POST_STACK_3D_JS_017534_tol1_JFA.vds
```
And the error I'm getting is:
```
[Could not create VDS sd://osdu/vdstestsjfa/test/ST0202R08_PS_PSDM_RAW_PP_TIME_MIG_RAW_POST_STACK_3D_JS_017534_tol1_JFA.vds] Error on uploading VolumeDataLayout object: Http error response: 301 -> https://psosdu-shared-seismicddms-20230814174725984500000004.s3.us-east-1.amazonaws.com/3o7c5j88s1ko0oyg/2b9b212b-21b5-4ccf-aaed-67485c113ae4/VolumeDataLayout: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
```
Seems to be accessing the wrong location since the instance is located at us-east-2.
Regards,
Juliana.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/209Adding CRS to a VDS generated by Openvds+2023-11-16T12:21:10ZJuliana Fernandesjuliana.fernandes@iesbrazil.com.brAdding CRS to a VDS generated by Openvds+Hello,
I was taking a look into the doccumentation in order to add CRS to the VDS I'm generating with Openvds+.
In the doccumentation I saw the command "–crs-wkt <string>". The WKT is a Well-known Text and seems to be a geographical c...Hello,
I was taking a look into the doccumentation in order to add CRS to the VDS I'm generating with Openvds+.
In the doccumentation I saw the command "–crs-wkt <string>". The WKT is a Well-known Text and seems to be a geographical coordinate. There is a way to add a UTM coordinate to the data?
Regards,
Julianahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/208Multiple queries of data will be much slower.2023-09-27T08:35:56Znanting liuMultiple queries of data will be much slower.Multiple queries of data at different depths then the response time of results will be much slower(from the first few tens of milliseconds to seven seconds)
_VolumeDataRequestDouble data= accessManager.requestVolumeSubsetDouble(managedBu...Multiple queries of data at different depths then the response time of results will be much slower(from the first few tens of milliseconds to seven seconds)
_VolumeDataRequestDouble data= accessManager.requestVolumeSubsetDouble(managedBuffer.getByteBuffer(), Dimensions_012, 0, 0, min, max);_
_data.waitForCompletion();_
_getdata()_
_managedBuffer.close();_
_vds.close();_
i found Loop call _requestVolumeSubsetDouble_ then each response time will be longer.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/207Is there any way to get min value of sample or max value?2023-09-21T10:03:07Znanting liuIs there any way to get min value of sample or max value?i use `VolumeDataAccessManager.requestVolumeSubsetDouble()` to get sample data, and then i want to get the maximum and minimum values in all data, so i have to write code to compare them, is there any usable method to invoke to get that ...i use `VolumeDataAccessManager.requestVolumeSubsetDouble()` to get sample data, and then i want to get the maximum and minimum values in all data, so i have to write code to compare them, is there any usable method to invoke to get that two values in _VolumeDataLayout_ or _VolumeDataAccessManager_?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/206SEGYImport will create a ramdon directory on my cloud environment?2023-09-20T18:51:48Znanting liuSEGYImport will create a ramdon directory on my cloud environment?l already defined a subpath with command("s3://vds/1704505448285212672"),but i found a record in the log("Successfully imported into s3://vds/1704505448285212672/CD352FB47324BA63"),how can I avoid this situation?l already defined a subpath with command("s3://vds/1704505448285212672"),but i found a record in the log("Successfully imported into s3://vds/1704505448285212672/CD352FB47324BA63"),how can I avoid this situation?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/205fail to copy file from cloud server2023-09-19T10:20:18Znanting liufail to copy file from cloud serveri try to read vds from minio(a open-source, S3 compatible object store)
at first step,i try to read a vds file with _OpenVDS.open()_,but failed,
exception is "Error on downloading VolumeDataLayout object: Http error response: 404 -> http...i try to read vds from minio(a open-source, S3 compatible object store)
at first step,i try to read a vds file with _OpenVDS.open()_,but failed,
exception is "Error on downloading VolumeDataLayout object: Http error response: 404 -> https://endpoint/bucket-name/test.vds/VolumeDataLayout: The specified key does not exist.".
then,i realized that _open()_ can not read a vds file directly, cause the file uploaded manually.
and the second step,l try to use VDSCopy to copy the VDS file to the cloud environment,still fail! with error "Error on uploading VolumeDataLayout object: unexpected AWS signing failure",here is my command `VDSCopy.exe E:\PPCoef.vds s3://endpoint/bucket-name/testVDS -d "Region=us-west-rack-2;SecretKey=xxx;SecretAccessKey=xxx"`,my SecretKey&SecretAccessKey is correct,but l dont know why print this...
Could you please help me figure out how to deal with this situation?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/204Add a mechanism to use an external application to get credentials2023-09-08T08:27:25ZMorten OfstadAdd a mechanism to use an external application to get credentialsIn order to get credentials that require a user to log in, it will be useful to run a separate executable. This is e.g. how git works (global config sets credential.helper to point to an executable, can be configured per URL prefix, see ...In order to get credentials that require a user to log in, it will be useful to run a separate executable. This is e.g. how git works (global config sets credential.helper to point to an executable, can be configured per URL prefix, see Git - gitcredentials Documentation (git-scm.com)). Integrating this directly in OpenVDS makes it easy for other applications to take advantage of.
The suggested implementation will add new global keys (valid for all cloud providers) credential_helper and credential_helper_args to the connection string format. If the credential_helper key is present, the executable pointed to will be run with the args from credential_helper_args and the URL as arguments, and the output will be parsed as a connection string and added to the remaining keys after removing the credential_helper and credential_helper_args keys. This allows other arguments like tolerance etc. to be passed on from the original connection string after using the credentials helper.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/203Exec SEGYIMPORT.exe(or .sh) is the only way to convert a .sgy file to .vds file?2023-09-08T08:21:10Znanting liuExec SEGYIMPORT.exe(or .sh) is the only way to convert a .sgy file to .vds file?l am confused that run SEGYIMPORT.exe(or .sh) is the only method to convert a .sgy file to .vds file?l am confused that run SEGYIMPORT.exe(or .sh) is the only method to convert a .sgy file to .vds file?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/202[docs][webpage] seems Python API docs are gone2023-09-15T11:30:27ZFilip Brzęk[docs][webpage] seems Python API docs are gone[Python API Link](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/python-api.html)
![image](/uploads/5fc6bed0cd3a145846b921ac6d2221c1/image.png)[Python API Link](https://osdu.pages.opengroup.org/platform/domain-data-mgmt-services/seismic/open-vds/python-api.html)
![image](/uploads/5fc6bed0cd3a145846b921ac6d2221c1/image.png)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/201SEG-Y header field definition at the last byte location doesn't work2023-09-06T10:11:02ZMorten OfstadSEG-Y header field definition at the last byte location doesn't workSEGYImport does not allow reading the last FourByte header word from the SEGY header.
If the crossline in the last header word, you can define a JSON file as follows:
```
{
"InlineNumber" : [ 233, "FourByte"],
"CrosslineNumber"...SEGYImport does not allow reading the last FourByte header word from the SEGY header.
If the crossline in the last header word, you can define a JSON file as follows:
```
{
"InlineNumber" : [ 233, "FourByte"],
"CrosslineNumber" : [ 237, "FourByte"]
}
```
This gives an ‘illegal field’ definition error because the code checks that 237 + 4 is no more than 240 which is off-by-one.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/200Any hint on generate fast slices?2023-08-30T13:35:01ZQiang FuAny hint on generate fast slices?I would like to read data inline by inline, crl by crl. I saw the documents said it could be done by fast slices, any hint how to handle it? Does segyimport support it?I would like to read data inline by inline, crl by crl. I saw the documents said it could be done by fast slices, any hint how to handle it? Does segyimport support it?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/199SEGYImport DataProvider for DMS needs to support chunked datasets2024-02-26T20:08:56ZMorten OfstadSEGYImport DataProvider for DMS needs to support chunked datasetsSEG-Y datasets in DMS uploaded with sdutil will have a default chunk size of 32MB, the DataProvider class needs to support this in order to successfully import the data. See this issue for details:
https://community.opengroup.org/osdu/p...SEG-Y datasets in DMS uploaded with sdutil will have a default chunk size of 32MB, the DataProvider class needs to support this in order to successfully import the data. See this issue for details:
https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/585#note_246359Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/198Failed to create 3D Prestack container on OSDU2023-08-29T14:40:24ZAnatoly YanchevskyFailed to create 3D Prestack container on OSDUI try to create 3D Prestack with 4 axis:
Sample/Trace(offset)/Crossline/Inline
Error occurred on call "OpenVDS::Create".
Message: "DimensionGroup 41 is not a valid dimension."
Stack: [call_stack.txt](/uploads/fb0ec722ccda7ec907d357aa4b0...I try to create 3D Prestack with 4 axis:
Sample/Trace(offset)/Crossline/Inline
Error occurred on call "OpenVDS::Create".
Message: "DimensionGroup 41 is not a valid dimension."
Stack: [call_stack.txt](/uploads/fb0ec722ccda7ec907d357aa4b03a211/call_stack.txt)
Same volume can be created on local disk.
Output from VDSInfo: [VDSInfo_axis.txt](/uploads/1ecb09ef3ce874ef18686d3397a45a61/VDSInfo_axis.txt) and [VDSInfo_channels.txt](/uploads/bd3daf2ce80b5022ecab479f329109d7/VDSInfo_channels.txt)
Probably, I do something wrong, but I did not understand what exactly.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/197Deadlock in VolumeDataRequestProcessor::WaitForCompletion after cancel loadin...2023-08-25T07:48:49ZAnatoly YanchevskyDeadlock in VolumeDataRequestProcessor::WaitForCompletion after cancel loading one of LODsWe create several LODs in OpenVDS container. When our viewer display data, some time need to cancel loading one of the LODs.
It produce deadlock in open-vds library.
Two threads that are in the deadlock: [stack_1.txt](/uploads/25ff1d9f8...We create several LODs in OpenVDS container. When our viewer display data, some time need to cancel loading one of the LODs.
It produce deadlock in open-vds library.
Two threads that are in the deadlock: [stack_1.txt](/uploads/25ff1d9f89d453a9b1a11eec99dcb5ab/stack_1.txt) and [stack_2.txt](/uploads/8dfb70257b51f09f3f6c8df050abdc5b/stack_2.txt)
To fix that, we do next changes: [VolumeDataRequestProcessor.patch](/uploads/9ddab5fc678d95834df15a07fc5874d9/VolumeDataRequestProcessor.patch)
Problem occurred on Linux, open-vds 3.0.3. But we found that in latest version function VolumeDataRequestProcessor::WaitForCompletion did not changed.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/196Wavelet compression in Python2023-08-29T14:41:08ZVasilii SinkevichWavelet compression in PythonHi,
I am investigating VDS capabilities and trying to convert numpy array into VDS using Wavelet compression.
I am using the example provided along the source code (npy_to_vds.py) with minimal changes
Everything works fine with no com...Hi,
I am investigating VDS capabilities and trying to convert numpy array into VDS using Wavelet compression.
I am using the example provided along the source code (npy_to_vds.py) with minimal changes
Everything works fine with no compression or with Zip or RLE compression, I can write and read data back, but if I try using Wavelet, like this:
```python
compressionMethod = openvds.CompressionMethod(1)
```
the code execution gets stuck at committing changes
```python
accessor.commit()
```
regardless of compression tolerance value. It just hangs here for ages.
I am wondering if python port is actual OpenVDS+ binding, maybe it is actually OpenVDS (without plus) that do not support wavelet compression?
If it *is* OpenVDS+, could you please provide a sample of the code (or maybe edit npy_to_vds.py) that saves VDS format with wavelet compression?
EDIT: I am working on Debian 9, python 3.9, openvds 3.2.6
Thanks,
Vasiliihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/195Missing const in Sample-functions in VolumeDataAxisDescriptor2023-07-04T09:13:40ZAlena ChaikouskayaMissing const in Sample-functions in VolumeDataAxisDescriptor```
SampleIndexToCoordinate
CoordinateToSampleIndex
CoordinateToSamplePosition
```
functions in [OpenVDS::VolumeDataAxisDescriptor](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/s...```
SampleIndexToCoordinate
CoordinateToSampleIndex
CoordinateToSamplePosition
```
functions in [OpenVDS::VolumeDataAxisDescriptor](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/src/OpenVDS/OpenVDS/VolumeDataAxisDescriptor.h#L113) seem to not modify anything, yet they are not marked as const.
Not being able to use const objects with these functions is inconvenient.
Is this an oversight or is it in theory possible for these operations to change something?