Seismic issueshttps://community.opengroup.org/groups/osdu/platform/domain-data-mgmt-services/seismic/-/issues2022-11-01T11:40:00Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/1513.0.4 fmt related build failure in IOManagerAzureSdkForCpp2022-11-01T11:40:00ZAlena Chaikouskaya3.0.4 fmt related build failure in IOManagerAzureSdkForCppHi,
We have a problem with building 3.0.4 with new azure sdk.
We are running
```
RUN cmake -S . \
-B build \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_JAVA=OFF \
-DBUILD_PYTHON=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TE...Hi,
We have a problem with building 3.0.4 with new azure sdk.
We are running
```
RUN cmake -S . \
-B build \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_JAVA=OFF \
-DBUILD_PYTHON=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_DOCS=OFF \
-DDISABLE_AWS_IOMANAGER=ON \
-DDISABLE_AZURESDKFORCPP_IOMANAGER=OFF \
-DDISABLE_GCP_IOMANAGER=ON \
-DDISABLE_DMS_IOMANAGER=OFF \
-DDISABLE_STRICT_WARNINGS=OFF
```
Unfortunately on all checked OS (alpine, etc) that fails with
```
In file included from /open-vds/3rdparty/fmt-9.1.0/include/fmt/format.h:48,
from /open-vds/src/OpenVDS/IO/IOManagerAzureSdkForCpp.h:33,
from /open-vds/src/OpenVDS/IO/IOManager.cpp:32:
/open-vds/3rdparty/fmt-9.1.0/include/fmt/core.h:1334:47: error: there are no arguments to 'format_as' that depend on a template parameter, so a declaration of 'format_as' must be available [-fpermissive]
```
If in the build script we exchange 9.1.0 with 7.1.3, build is run fine.
But then on usage new errors appear:
```
libazure-core.so, needed by /open-vds/Dist/OpenVDS/lib/libopenvds.so, not found (try using -rpath or -rpath-link)
libazure-storage-blobs.so, needed by /open-vds/Dist/OpenVDS/lib/libopenvds.so, not found (try using -rpath or -rpath-link)
libazure-storage-common.so, needed by /open-vds/Dist/OpenVDS/lib/libopenvds.so, not found (try using -rpath or -rpath-link)
```
That makes sense as only some azure libraries are copied:
```
Installing: /open-vds/Dist/OpenVDS/lib/libazurestorage.so.7.5
Installing: /open-vds/Dist/OpenVDS/lib/libazurestorage.so.7
```
when before (code based on version 2.3.3) `libazure-core.so`, `libazure-storage-blobs.so` and `libazure-storage-common.so` were installed as well.
```
Installing: /open-vds/Dist/OpenVDS/lib/libazurestorage.so.7.5
Installing: /open-vds/Dist/OpenVDS/lib/libazurestorage.so.7
Installing: /open-vds/Dist/OpenVDS/lib/libazurestorage.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-core.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-storage-common.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-template.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-identity.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-security-keyvault-common.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-security-keyvault-keys.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-storage-blobs.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-storage-files-datalake.so
Installing: /open-vds/Dist/OpenVDS/lib/libazure-storage-files-shares.so
```
Now to work around this we have to manually copy `azure-sdk-for-cpp_12.3.0_install` artifacts to installation directory.
Can this be fixed?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/1532bit float to 16bit integer for ZGY output2021-02-26T01:11:26ZTerry Walters32bit float to 16bit integer for ZGY outputI've not done this before so I'm basing this on research I've done on the web.
To scale 32bit float to store as 16bit integer I am planning to:
determine the maximum 32bit float value in the dataset.
scale each sample by dividing it by...I've not done this before so I'm basing this on research I've done on the web.
To scale 32bit float to store as 16bit integer I am planning to:
determine the maximum 32bit float value in the dataset.
scale each sample by dividing it by the maximum 32bit float, multiplying it by 32767, and truncating it.
sample[x] = (int16_t)(sample[x] / maxfloat) * 32767)
Does this make sense or am I way off base?
Thanks,
Terry WaltersPaal KvammePaal Kvammehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/1053D visualization demo using Python FastAPI, Three.js. Error when include lib ...2022-01-05T09:58:53ZTrung Dang3D visualization demo using Python FastAPI, Three.js. Error when include lib in C++.I've worked on a simple demo to retrieve slices data and plot 3 type of slices using Three.js which is available [here](https://github.com/trungdang97/openvds-threejs).
![image](/uploads/2d890c3fe67ef220995bbadff3923cf8/image.png)
I do...I've worked on a simple demo to retrieve slices data and plot 3 type of slices using Three.js which is available [here](https://github.com/trungdang97/openvds-threejs).
![image](/uploads/2d890c3fe67ef220995bbadff3923cf8/image.png)
I don't have access to any cloud S3 (I've tried Minio Playground but it's terribly slow) so I used VDS file instead for the demo - which is imported from Kerry3D data. It works fine but the response time is slow (between 2-3 seconds for a slice) so the transition is not smooth.
I'm trying to create a REST service using C++ but I'm not familiar with the language. I copied these 2 folder (OpenVDS, SEGYUtils) from OpenVDS+2.1.8 distribution into my include folder and proceed copy the C++ example. But there was an issue with undefined error from the variable "volumeDataAccessManagerInterface" at line 944 in OpenVDS.h.
`
g++ -std=c++11 -fdiagnostics-color=always -g W:\VDS\vds-3d-plot\cpp\api.cpp -o W:\VDS\vds-3d-plot\cpp\api.exe -I W:\VDS\vds-3d-plot\cpp\include
`
`C:\Users\Trung\AppData\Local\Temp\ccGYBsSs.o: In function 'main':
W:/VDS/vds-3d-plot/cpp/api.cpp:22: undefined reference to '__imp__ZN7OpenVDS4OpenENS_13StringWrapperES0_RNS_5ErrorE'
`
`C:\Users\Trung\AppData\Local\Temp\ccGYBsSs.o: In function
'GetAccessManager':
W:/VDS/vds-3d-plot/cpp/include/OpenVDS/OpenVDS.h:944: undefined reference to '__imp__ZN7OpenVDS25GetAccessManagerInterfaceEPNS_3VDSE
'`
`
collect2.exe: error: ld returned 1 exit status
`
Can someone help me with the error? I know this is not Stack Overflow but if someone can show me a minimal working example using OpenVDS in C++ so I can figure out how it works and continue doing experiments on decreasing the service response time. Thank you in advance.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/35500 errors during creating subproject2021-12-13T12:56:30ZVladislav Shishko (EPAM)500 errors during creating subproject`curl --location --request POST 'https://seismic-store-drgfbg5txq-uc.a.run.app/api/v3/subproject/tenant/autotestTenantid911741/subproject/subprojectodi864398' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: ode...`curl --location --request POST 'https://seismic-store-drgfbg5txq-uc.a.run.app/api/v3/subproject/tenant/autotestTenantid911741/subproject/subprojectodi864398' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: odesprod' \
--header 'ltag;' \
--header 'Authorization: Bearer ****' \
--data-raw '{
"admin": "admin@odesprod.osdu-gcp.go3-nrg.projects.epam.com",
"storage_class": "MULTI_REGIONAL",
"storage_location": "US",
"acls": {
"admins": [
"data.sdms.autotestTenantid911741.subprojectodi864398.admin@odesprod.osdu-gcp.go3-nrg.projects.epam.com"
],
"viewers": [
"data.sdms.autotestTenantid911741.subprojectodi864398.viewer@odesprod.osdu-gcp.go3-nrg.projects.epam.com"
]
},
"legal": {
"legaltags": [
"odesprod-Test-Legal-Tag-3354434"
],
"otherRelevantDataCountries": [
"US"
]
}
}'`
expected 200M9 - Release 0.12https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/15502 Gateway Error on calling Redis get and set functions2021-02-23T04:27:47ZWalter D502 Gateway Error on calling Redis get and set functionsHi @DiegoMolteni
We have been getting the 502 Gateway Error for some APIs. One of the APIs is Create Subproject API. On debugging the code the error is thrown on the following 2 lines in compliance.ts file in the create subproject flow:...Hi @DiegoMolteni
We have been getting the 502 Gateway Error for some APIs. One of the APIs is Create Subproject API. On debugging the code the error is thrown on the following 2 lines in compliance.ts file in the create subproject flow:
1. await this._cache.set(ltag, results.invalidLegalTags.length === 0);
2. await this._cache.set(ltag, results.invalidLegalTags.length === 0);
Interestingly, this is happening on the DEV environment. I've not encountered the issue in my local. Have you faced this error or have a clue on what the problem could be? Any help would be appreciated. Thank you.M1 - Release 0.1Diego MolteniDaniel PerezDiego Molteni2021-02-26https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/167Abort trap 6 on OpenVDS::Open2023-08-29T14:42:46ZErlend HårstadAbort trap 6 on OpenVDS::OpenPassing `OpenVDS::Open` a sas token where the Signed Resource Type (`srt`) parameter contains 'Container' (`c`) and _not_ object (`o`) cause an Abort trap 6. As far as I can tell, this happen for any combination of `srt` options as long ...Passing `OpenVDS::Open` a sas token where the Signed Resource Type (`srt`) parameter contains 'Container' (`c`) and _not_ object (`o`) cause an Abort trap 6. As far as I can tell, this happen for any combination of `srt` options as long as it contains `c` and not `o`. Although this particular combination doesn't make a whole lot of sense when working with VDS, it's still a completely valid sas.
I've only tested this on 3.0.3https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/issues/3Access dataset in seismi-drive via cloud url/credentials2021-06-07T19:20:09ZRaghu Jayan MenonAccess dataset in seismi-drive via cloud url/credentialsHello,
Is there a way to:
1. Get the fully qualified cloud url for the backing cloud provider for a dataset eith via API and through REST endpoint to the service.
2. Is there a mechanism to generate/refresh credentials that can work wit...Hello,
Is there a way to:
1. Get the fully qualified cloud url for the backing cloud provider for a dataset eith via API and through REST endpoint to the service.
2. Is there a mechanism to generate/refresh credentials that can work with (1)
I have tried to use the gcsUrl API however, it does not for example provide account_name in case of Azure and the SDToken cannot be used outside SDAPI (I assume).
Thank you,
Raghuhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/17Add a benchmarking tool2020-03-10T10:28:09ZMorten OfstadAdd a benchmarking toolThe VDSBenchmark should have options to read the whole dataset as slices (along any axis) and tiles, control how many requests are in the queue and size of tiles. It should report detailed timings and MB/s rate.The VDSBenchmark should have options to read the whole dataset as slices (along any axis) and tiles, control how many requests are in the queue and size of tiles. It should report detailed timings and MB/s rate.Jørgen Lindjorgen.lind@3lc.aiJørgen Lindjorgen.lind@3lc.aihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/204Add a mechanism to use an external application to get credentials2023-09-08T08:27:25ZMorten OfstadAdd a mechanism to use an external application to get credentialsIn order to get credentials that require a user to log in, it will be useful to run a separate executable. This is e.g. how git works (global config sets credential.helper to point to an executable, can be configured per URL prefix, see ...In order to get credentials that require a user to log in, it will be useful to run a separate executable. This is e.g. how git works (global config sets credential.helper to point to an executable, can be configured per URL prefix, see Git - gitcredentials Documentation (git-scm.com)). Integrating this directly in OpenVDS makes it easy for other applications to take advantage of.
The suggested implementation will add new global keys (valid for all cloud providers) credential_helper and credential_helper_args to the connection string format. If the credential_helper key is present, the executable pointed to will be run with the args from credential_helper_args and the URL as arguments, and the output will be parsed as a connection string and added to the remaining keys after removing the credential_helper and credential_helper_args keys. This allows other arguments like tolerance etc. to be passed on from the original connection string after using the credentials helper.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/issues/13Add a possibility to work with Anthos/MinIO.2023-03-30T16:59:36ZYan Sushchynski (EPAM)Add a possibility to work with Anthos/MinIO.Hello!
We implemented Seismic DMS for `Anthos` environment with MinIO as a storage backend. MinIO implementation works with `s3` and mostly follows AWS implementation.
For `sdutil`, all we needed was overriding a single method of `AwsS...Hello!
We implemented Seismic DMS for `Anthos` environment with MinIO as a storage backend. MinIO implementation works with `s3` and mostly follows AWS implementation.
For `sdutil`, all we needed was overriding a single method of `AwsStorageService` service; we just added MinIO endpoint.
(https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/merge_requests/50/diffs#316b48789c54e1de5292718db6abac8856b6cec3_0_48)
The problem is that `Open VDS` and `Open ZGY` convertors use `seismic-cpp-lib` and there is no way to access MinIO storage to manipulate files.
I found that the library gets the information about Cloud provider for choosing a storage class from the response header of Seismic
(https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/blob/master/src/src/lib/cloud/SeismicStore.cc#L704).
This value is taken from `CLOUDPROVIDER` env var of Seismic deployment. As our `CLOUDPROVIDER`'s value is `anthos`, `seismic-cpp-lib` chooses `GcsAccessorStorage`(
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/blob/master/src/src/lib/cloud/Storage.cc#L172)
What we need:
1. Choose `AwsStorage` if cloud provider is `anthos`;
1. Possibility to override AWS's Endpoint URL with the environmental variable's value.
SDMS MR:
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/322
Thanks.
fyi: @Siarhei_Khaletski @jorgenhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/71Add command-line parameters to SEGYImport tool for overriding specific header...2021-03-03T15:41:36ZMorten OfstadAdd command-line parameters to SEGYImport tool for overriding specific header fieldsWhen SEGYImport is being called from a script/in a container it can be inconvenient to create a header format JSON file, so we should provide the option to override header fields directly on the command line.When SEGYImport is being called from a script/in a container it can be inconvenient to create a header format JSON file, so we should provide the option to override header fields directly on the command line.Version 2.0https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/220Add CRS database lookup to SEGYImport tool2023-11-16T12:21:10ZMorten OfstadAdd CRS database lookup to SEGYImport toolIt would be so much easier if it was possible to specify CRS by using the EPSG ID or UTM zone to look up the CRSWkt automatically, e.g. like this: --crs EPSG:23031 or --crs UTM-31N to get the WKT
```
PROJCS["ED50 / UTM zone 31N",GEOGCS["...It would be so much easier if it was possible to specify CRS by using the EPSG ID or UTM zone to look up the CRSWkt automatically, e.g. like this: --crs EPSG:23031 or --crs UTM-31N to get the WKT
```
PROJCS["ED50 / UTM zone 31N",GEOGCS["ED50",DATUM["European_Datum_1950",SPHEROID["International 1924",6378388,297],TOWGS84[-87,-98,-121,0,0,0,0]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4230"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",3],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","23031"]]
```
I looked up this at [https://epsg.io/23031](https://epsg.io/23031)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/68Add exception translation to the Java API2022-07-21T13:12:49ZMorten OfstadAdd exception translation to the Java APIThe native VolumeDataAccessManager will throw exceptions for bad parameters etc., these should be translated to standard Java exceptions.The native VolumeDataAccessManager will throw exceptions for bad parameters etc., these should be translated to standard Java exceptions.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/209Adding CRS to a VDS generated by Openvds+2023-11-16T12:21:10ZJuliana Fernandesjuliana.fernandes@iesbrazil.com.brAdding CRS to a VDS generated by Openvds+Hello,
I was taking a look into the doccumentation in order to add CRS to the VDS I'm generating with Openvds+.
In the doccumentation I saw the command "–crs-wkt <string>". The WKT is a Well-known Text and seems to be a geographical c...Hello,
I was taking a look into the doccumentation in order to add CRS to the VDS I'm generating with Openvds+.
In the doccumentation I saw the command "–crs-wkt <string>". The WKT is a Well-known Text and seems to be a geographical coordinate. There is a way to add a UTM coordinate to the data?
Regards,
Julianahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/14Add IOManagerCurl2021-03-11T12:41:38ZJørgen Lindjorgen.lind@3lc.aiAdd IOManagerCurlTo maintain a predictable performance between IOManagers there should exist a IOManager utilising curl on both Windows and Linux that communicates with a server without any authentication. This can then be referenced as a benchmark for p...To maintain a predictable performance between IOManagers there should exist a IOManager utilising curl on both Windows and Linux that communicates with a server without any authentication. This can then be referenced as a benchmark for performance characteristics the other IOManagers should achieve. The IOManager should be written in such a way that it should be "trivial" for other IOManagers to utilise the http transfer code and add its own signing headers etc.Version 2.0Jørgen Lindjorgen.lind@3lc.aiJørgen Lindjorgen.lind@3lc.aihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/174Addition of valid input for SEGYImport2023-02-23T13:23:58ZAlexander JaustAddition of valid input for SEGYImportI wonder what would be the correct the conversion of SEG-Y files to VDS using `SEGYImport` when the provided options by `SEGYImport` are not rich enough.
Example: I have an attribute map in a SEG-Y file. However, none of the allowed na...I wonder what would be the correct the conversion of SEG-Y files to VDS using `SEGYImport` when the provided options by `SEGYImport` are not rich enough.
Example: I have an attribute map in a SEG-Y file. However, none of the allowed names for the `Attribute` property (`--attribute-name`: Amplitude (default), Attribute, Depth, Probability, Time, Vavg, Vint, or Vrms) are fitting my needs. The attribute name `Attribute` would be too vague for my use case and the other allowed names do not fit either.
- Should I import the SEG-Y file to VDS and afterwards change the name? I am not sure if that is possible, cf. #173.
- Could I provide a patch that extends SEGYImport by the units, names etc. that I would need?
- Should I fork and create my own SEGYImport? I would really like to avoid this.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/41Add library path instructions to readme2021-06-16T22:19:43ZJørgen Lindjorgen.lind@3lc.aiAdd library path instructions to readmehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/41Add new version info endpoint2023-06-13T20:06:33ZSiarhei Khaletski (EPAM)Add new version info endpointOriginal ADR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/47
Additional info: https://community.opengroup.org/osdu/platform/home/-/issues/36Original ADR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/47
Additional info: https://community.opengroup.org/osdu/platform/home/-/issues/36https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/64Add options for adaptive streaming2021-03-05T10:41:04ZMorten OfstadAdd options for adaptive streamingAll the code to do the adaptive requests and decompression is there, we need to add the options to actually use it...All the code to do the adaptive requests and decompression is there, we need to add the options to actually use it...Version 2.0Morten OfstadMorten Ofstadhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/190Add OSDU manifest generation capability2023-06-02T14:03:18ZMorten OfstadAdd OSDU manifest generation capabilityVDSInfo (or maybe a new vds utility) should be able to generate OSDU compliant JSON manifest for ingesting VDS. This was suggested by Juliana Fernandes (@fernandes_jfa) and would make life a lot easier when interacting with the OSDU inge...VDSInfo (or maybe a new vds utility) should be able to generate OSDU compliant JSON manifest for ingesting VDS. This was suggested by Juliana Fernandes (@fernandes_jfa) and would make life a lot easier when interacting with the OSDU ingestion service.