Open ZGY issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues2020-10-26T11:35:44Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/3Sample zgy file in Testdata folder2020-10-26T11:35:44ZDebasis ChatterjeeSample zgy file in Testdata folder@kvamme3 - Can you please provide some description about available test data sets?
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/tree/master/testdata
The names include "empty" which leads t...@kvamme3 - Can you please provide some description about available test data sets?
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/tree/master/testdata
The names include "empty" which leads the reader to think that these do not contain any real data.
Perhaps one good example would be a simple small SegY file and matching ZGY file plus any supplementary information (presumably corner point inline/crossline, X, Y, CRS and whatever else) that is required over and above the raw SegY to make up the Zgy file.Paal KvammePaal Kvammehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/5Looking for clear API documentation - write Zgy2020-11-05T01:27:46ZDebasis ChatterjeeLooking for clear API documentation - write ZgyIssue created on behalf of my colleague Terry Walters.
Coding in C++
Looking for clear documentation about what exactly is needed to create a Zgy output file.
Something similar to
(A) SegY specification which someone can take to buil...Issue created on behalf of my colleague Terry Walters.
Coding in C++
Looking for clear documentation about what exactly is needed to create a Zgy output file.
Something similar to
(A) SegY specification which someone can take to build SegY file from scratch, with trace data and relevant information in various headers.
(B) Actual API documentation explaining what's needed to create a ZGY output file
Thank youhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/4Simple "Hello World" style program to read ZGY file by using the library2020-11-08T20:32:56ZDebasis ChatterjeeSimple "Hello World" style program to read ZGY file by using the library@imonteiro - Following up from your recent overview session, I was checking in wiki repository for any such sample code. Do you provide such example for reading a Zgy? This will help first time user for a quick understanding and will als...@imonteiro - Following up from your recent overview session, I was checking in wiki repository for any such sample code. Do you provide such example for reading a Zgy? This will help first time user for a quick understanding and will also help OSDU "Data Loading" team to be able to make up a load manifest if there is a desire to ingest ZGY file in OSDU, similar to SegY for SeismicTraceData work-product-component.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/7Looking for clear description of the Zgy format2020-11-17T11:37:40ZDebasis ChatterjeeLooking for clear description of the Zgy formatSo far, I have found two documents in DOC folder.
- openzgy.html
- openzgy-fig1.png
Some acronyms (LOD, LUP, LUT) are not expanded.
Can you please point me to some other file(s), if they exist, which provide clear documentation about ...So far, I have found two documents in DOC folder.
- openzgy.html
- openzgy-fig1.png
Some acronyms (LOD, LUP, LUT) are not expanded.
Can you please point me to some other file(s), if they exist, which provide clear documentation about Zgy file structure?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/9Pre-requisite steps for Linux development environment - to do any work using ...2020-11-22T13:05:45ZDebasis ChatterjeePre-requisite steps for Linux development environment - to do any work using OSDU/Openzgy@kvamme3 - As I await instructions in Windows environment (Issue#8), can you please provide list of per-requisites to run one of the independent module (such as zgydump) in Linux?
- I have access to Python 3.4.5 in Linux environment
- T...@kvamme3 - As I await instructions in Windows environment (Issue#8), can you please provide list of per-requisites to run one of the independent module (such as zgydump) in Linux?
- I have access to Python 3.4.5 in Linux environment
- Tested simple Python program in that environment
- List of packages to install
- Is there anything else I need to bring from OSDU/Openzgy? Or can I simply get the zgydump.py file and execute after installing required packages?
I am trying to run zgydump in order to list content of a Zgy file built elsewhere.
Thank youhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/6Suitable place in Zgy structure for contents equivalent of Text and Binary he...2020-12-05T09:20:15ZDebasis ChatterjeeSuitable place in Zgy structure for contents equivalent of Text and Binary header from SegYJust keeping SegY as reference for content of Seismic trace data, we are wondering what would be suitable spots for this kind of information (Text header, Binary header) if we are trying to use the library to produce a Zgy file.
Thanks ...Just keeping SegY as reference for content of Seismic trace data, we are wondering what would be suitable spots for this kind of information (Text header, Binary header) if we are trying to use the library to produce a Zgy file.
Thanks for your help.
Debasis
cc - Terry Waltershttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/10Is compression supported by C++ library?2020-12-14T16:23:31ZDebasis ChatterjeeIs compression supported by C++ library?Please indicate if compression is supported when writing Zgy file by using C++ library? If yes, provide suitable pointer to relevant information. Thank you.Please indicate if compression is supported when writing Zgy file by using C++ library? If yes, provide suitable pointer to relevant information. Thank you.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/11Requesting information about how to make estimation of size of Zgy during wri...2020-12-21T22:42:35ZDebasis ChatterjeeRequesting information about how to make estimation of size of Zgy during write (uncompressed, compressed)Taking simple example of 3D trace data.
To give an idea, each trace has 5501 samples of 4 bytes each. Total of 4395 traces.
SegY filesize is 93.2 MB.
3200+400+(240+5501*4)*4395 = 97765980 bytes.
As you can see, I am using 3200 bytes f...Taking simple example of 3D trace data.
To give an idea, each trace has 5501 samples of 4 bytes each. Total of 4395 traces.
SegY filesize is 93.2 MB.
3200+400+(240+5501*4)*4395 = 97765980 bytes.
As you can see, I am using 3200 bytes for EBCDIC header, 400 bytes for Binary header and 240 bytes for each trace header.
If we put the same data in Zgy format, what should the size of Zgy be - when uncompressed and when compressed?
Your response will be good reference for us to determine if we are using WRITE feature of C++ library properly.
Thank you in advancehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/12Sample zgy file in Openzgy library -2020-12-22T14:20:29ZDebasis ChatterjeeSample zgy file in Openzgy library -File = testdata_Fancy-int8.zgy
When I use sample code to get a summary overview, I find missing information for one line as shown below -
```
Horizontal projection system =
Horizontal dim/factor/name = length 0.3048 'ft'
```
Is ...File = testdata_Fancy-int8.zgy
When I use sample code to get a summary overview, I find missing information for one line as shown below -
```
Horizontal projection system =
Horizontal dim/factor/name = length 0.3048 'ft'
```
Is there a chance that you can replace the file with another zgy which contains entry for Horizontal projection system?
Thank you
[zgy-info-testdata_fancy-int8-new.txt](/uploads/7c4869fe571c2f373cef84d8fd2517f9/zgy-info-testdata_fancy-int8-new.txt)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/13Expected treatment of 2D data?2020-12-28T12:46:48ZDebasis ChatterjeeExpected treatment of 2D data?The documentation mentions Inline, Crossline. So, I am interpreting that this is geared towards 3D data. Can you please give some idea about coverage of 2D data (SP, CDP etc.)? Thank youThe documentation mentions Inline, Crossline. So, I am interpreting that this is geared towards 3D data. Can you please give some idea about coverage of 2D data (SP, CDP etc.)? Thank youhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/14Issues when creating Zgy from SegY (C++ library)2021-01-29T09:54:03ZDebasis ChatterjeeIssues when creating Zgy from SegY (C++ library)Submitting on behalf of Terry Walters
I'm attempting to create a ZGY file from SEGY using C++ and have a few issues.
- I'm writing individual SEGY traces to ZGY using the method: write(const size3i_t& start, const size3i_t& size, const...Submitting on behalf of Terry Walters
I'm attempting to create a ZGY file from SEGY using C++ and have a few issues.
- I'm writing individual SEGY traces to ZGY using the method: write(const size3i_t& start, const size3i_t& size, const float* data) from the class OPENZGY_API IZgyWriter. When I am finished and call finalize and close I expect everything to be flushed to disk. This does not appear to be the case. When I read the data back everything checks out until what appears to be the last brick. Any ideas?
- All of my traces appear to be being stored in 16384 byte blocks even though they are only 8000 bytes in length. Do I need to be building bricks and writing them instead of writing individual traces?
- This goes back to building bricks instead of writing individual traces; I also need to have the option to add ZFP compression. This appears to be straight forward enough but throws an error: Updating a local BrickStatus::3 brick with compressed data is illegal in UpdateMode::1 . I assume, again, that this means I need to be writing bricks instead of traces?
Any help is appreciated.
Thank you
Debasishttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/1532bit float to 16bit integer for ZGY output2021-02-26T01:11:26ZTerry Walters32bit float to 16bit integer for ZGY outputI've not done this before so I'm basing this on research I've done on the web.
To scale 32bit float to store as 16bit integer I am planning to:
determine the maximum 32bit float value in the dataset.
scale each sample by dividing it by...I've not done this before so I'm basing this on research I've done on the web.
To scale 32bit float to store as 16bit integer I am planning to:
determine the maximum 32bit float value in the dataset.
scale each sample by dividing it by the maximum 32bit float, multiplying it by 32767, and truncating it.
sample[x] = (int16_t)(sample[x] / maxfloat) * 32767)
Does this make sense or am I way off base?
Thanks,
Terry WaltersPaal KvammePaal Kvammehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/16Read and write compressed ZGY2021-05-04T18:46:48ZTerry WaltersRead and write compressed ZGYI'm looking for information on how to read a ZFP compressed ZGY file.
Writing compressed seems fairly simple. Just add .zfp_compressor(30) and .zfp_lodcompressor(50) to the IZgyWriterArgs and it appears to compress the output.
I have ...I'm looking for information on how to read a ZFP compressed ZGY file.
Writing compressed seems fairly simple. Just add .zfp_compressor(30) and .zfp_lodcompressor(50) to the IZgyWriterArgs and it appears to compress the output.
I have not found any comments or examples on how to read back the compressed ZGY file.
Can you point me in the right direction?https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/17Latest version versus Salmon2021-05-19T11:44:16ZTerry WaltersLatest version versus SalmonI recently wrote an app to store SEG-Y data in OpenZGY using the latest library downloads.
When a client attempts to read my ZGY output the get an error message showing, in part, the following:
Technical details: Failed to open ZGY fil...I recently wrote an app to store SEG-Y data in OpenZGY using the latest library downloads.
When a client attempts to read my ZGY output the get an error message showing, in part, the following:
Technical details: Failed to open ZGY file - file version is newer than this program
Type of exception: Slb::Salmon::BasePlugins::SalmonShared::ZgyAccess::ZgyException
Is this to be expected? The older Salmon libraries not being able to read the current OpenZGY libraries output?
I also have an app, using the same latest library downloads, that lets me read my output.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/18.NET bindings2021-06-24T10:14:51ZRobert Schmidt.NET bindingsWe ([Cegal](https://cegal.com/)) have a branch with basic C and .NET bindings to the (new) OpenZGY API.
- netstandard2.0, with xunit tests and a console app for net5.0
- Capable of building Debug and Release nuget packages
- Console app...We ([Cegal](https://cegal.com/)) have a branch with basic C and .NET bindings to the (new) OpenZGY API.
- netstandard2.0, with xunit tests and a console app for net5.0
- Capable of building Debug and Release nuget packages
- Console app can dump ZGY meta data, copy and compare ZGY content
- Windows only for now (I would need assistance for Linux support)
- Seismic Store support - sdapi binaries and headers are added to branch
- 2 source files are added to OpenZGY.vcxproj to provide the C bindings, no changes to existing code
- Bindings expose a simplified API for now (e.g. no compression or const support)
Would this be of interest for this repo?
Until more complete, it should probably live as a separate branch.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/8Looking for clear instructions to set up Python development environment in Wi...2021-08-06T12:33:26ZDebasis ChatterjeeLooking for clear instructions to set up Python development environment in Windows OSMy interest is to get started by checking simple programs from your distribution. Such as zgydump.py to read content of a Zgy file.
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/tree/master/p...My interest is to get started by checking simple programs from your distribution. Such as zgydump.py to read content of a Zgy file.
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/tree/master/python/openzgy/tools
Can you please tell me pre-requisite steps (over and above things like "pip install numpy" etc.) to execute this program from Windows environment, by using IDLE for example?
(Is Visual Studio 2019 mandatory? I have Python 3.8.5 installed and can do some simple development using IDLE.)
How to resolve "parent package" issues for "from ..api import ZgyReader, SampleDataType", "from ..test.utils import SDCredentials"? What are the pre-requisite steps?
Many thanks for your help.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/2missing zfp.h + libcurl and SDAPI nuget packages not found2021-09-28T11:27:04ZStephen Gallantmissing zfp.h + libcurl and SDAPI nuget packages not foundHello, under a windows development environment, when pulling this repo and opening the visual studio solution file:
open-zgy\native\windows\OpenZGY.sln
The code will not compile complaining about missing nuget packages:
libcurl v7.66.0....Hello, under a windows development environment, when pulling this repo and opening the visual studio solution file:
open-zgy\native\windows\OpenZGY.sln
The code will not compile complaining about missing nuget packages:
libcurl v7.66.0.1
libcurl.redist v7.66.0.1
SDAPI.Native.VS141 v2.15.525
SDAPI.Native.VS141.redist v2.15.525
These nuget packages cannot be found online.
Manually removing them from OpenZGY.vcxproj gets around the issue so the HAVE_SD flag may not be getting handled properly.
However there is another issue where it complains about missing zfp.h within:
open-zgy\native\src\impl\compress_zfp.cpp
Which is referenced from OpenZGY.vcxproj
Not clear but it appears build-zfp.bat within ZFP.vcxproj may not be working properly giving an MSB8065 warning:
Custom build for item "build-zfp.bat" succeeded, but specified output "...\open-zgy\build\deploy\native\x64\debug\zfp.dll" has not been created.
I would like to compile this from a windows development environment if possible?
Or if C++ binaries can be made available?? then that may be sufficient for us to use.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/20SEGY to ZGY conversion on Azure platform fails.2021-10-19T18:35:02ZKamlesh TodaiSEGY to ZGY conversion on Azure platform fails.The postman collection that executes successfully on AWS, IBM fails on Azure. In the Airflow log, the following error messages are seen - while registering DAG RBAC: access denied and BOOTSTRAP -- ERROR.
There are multiple segy-to-zgy-co...The postman collection that executes successfully on AWS, IBM fails on Azure. In the Airflow log, the following error messages are seen - while registering DAG RBAC: access denied and BOOTSTRAP -- ERROR.
There are multiple segy-to-zgy-conversion DAGs, I tried all of them. The log that is attached is using segy-to-zgy-conversion-0.10.0
Using the sdutil, I verified that the segy file is uploaded.
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil config init
[1] azure
Select the cloud provider: 1
Insert the azure (azureEnv) application key:
sdutil successfully configured to use azure (azureEnv)
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil auth login
Successfully logged into Azure SDUTIL.
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://opendes/autotesting
ST10010ZC11_PZ_PSDM_KIRCH_FULL_D.MIG_FIN.POST_STACK.3D.JS-017536.segy
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>
I have attached the postman collection and the airflow log[azure_segyToZgyConversionAirflowLog.txt](/uploads/1217079cd3f2f8c727474e0211ae24e2/azure_segyToZgyConversionAirflowLog.txt)
[SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json](/uploads/8836c93e8be06d3e7e24e33b55e0b9a3/SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json)
The environment file can be gotten from Platform Validation project and the following variables need to be added
segyToZgyWorkflowName = segy-to-zgy-conversion-0.10.0
subProjectName = autotesting
segyFileName = ST10010ZC11_PZ_PSDM_KIRCH_FULL_D.MIG_FIN.POST_STACK.3D.JS-017536.segyM9 - Release 0.12MANISH KUMARMANISH KUMARhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/19SEGY_TO_ZGY DAG does not return the correct status of the DAG when getting ...2021-10-19T18:44:31ZKamlesh TodaiSEGY_TO_ZGY DAG does not return the correct status of the DAG when getting the details for a specific workflow runwhen using the GET /v1/workflow/{workflow_name}/workflowRun/{runId} the SEGY_TOZGY DAG does not return the correct status.
It always returns the details with status = "submitted"even when the DAG has finished the execution. Which can be ...when using the GET /v1/workflow/{workflow_name}/workflowRun/{runId} the SEGY_TOZGY DAG does not return the correct status.
It always returns the details with status = "submitted"even when the DAG has finished the execution. Which can be verified by looking at the Airflow dashboard for the DAG.M9 - Release 0.12GregGreghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy/-/issues/21SEGY to ZGY conversion on GCP platform fails. [GONRG-3630]2021-10-20T15:51:13ZKamlesh TodaiSEGY to ZGY conversion on GCP platform fails. [GONRG-3630]The postman collection that executes successfully on AWS, IBM fails on GCP.
Using the sdutil, the file was uploaded or copied to the created subproject for the test.
Following documentation was used to follow the steps
[text](https://com...The postman collection that executes successfully on AWS, IBM fails on GCP.
Using the sdutil, the file was uploaded or copied to the created subproject for the test.
Following documentation was used to follow the steps
[text](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/blob/master/doc/testing.md#test-data)
D:\OSDU\dev\seismic-store-sdutil-master>python -m venv sdutilenv
D:\OSDU\dev\seismic-store-sdutil-master>sdutilenv\Scripts\activate.bat
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil config init
[1] google
Select the cloud provider: 1
Insert the google (defaultEnv) application key:
sdutil successfully configured to use google (defaultEnv)
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>set ID_TOKEN="Get this from postman by executing the refresh_token request"
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod --idtoken=%ID_TOKEN%
autotesting
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil stat sd://odesprod/autotesting --idtoken=%ID_TOKEN%
- Uri: sd://odesprod/autotesting
- Legal Tag: odesprod-Wellbore-Legal-Tag-Test3816976
- Storage Class: REGIONAL
- Storage Location: US-CENTRAL1
- Access Policy: uniform
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod/autotesting --idtoken=%ID_TOKEN%
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil cp data/ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy sd://odesprod/autotesting/ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy --idtoken=%ID_TOKEN%
- Initializing transfer session ... OK
- Initializing resumable-transfer location ... OK
- Uploading Data [ 100% |██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 0.98G/0.98G - 12:23|00:00 - 1.41MB/s ]
- Transfer completed: 1.347307686813347 [MB/s]
(sdutilenv) D:\OSDU\dev\seismic-store-sdutil-master>python sdutil ls sd://odesprod/autotesting --idtoken=%ID_TOKEN%
ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-017536.segy
The attached postman collection was used to ingest and convert the segy file to zgy.
A fully manual method (Way 1) is used in the collection where the following records are created FileCollection.SEGY, SeismicBinGrid, SeismicTraceData, and work-product. The document has to workflow service v1 & v2. Tried both
When version v2 was used got the message of endpoint/service was not found.
Even the doc the example for workflow service v2 shows it using v1 (`https://{path}/api/workflow/**v1**/workflow/{workflow-id}`
The airflow log does not give more detail other than ERROR - Task failed with exception.
This is the body content of the trigger request
{
"executionContext": {
"sd_svc_api_key": "",
"storage_svc_api_key": "",
"filecollection_segy_id": "{{file-collection-id}}",
"work_product_id": "{{work-product-id}}",
"data_partition_id": "{{data-partition-id}}",
"id_token": "{{id_token}}"
}
}
where various ids are saved from the manual steps executed prior to executing the trigger of the workflow request.
Attached are the postman collection used and the airflow log [AirflowLogOfFailedzgyComversion.txt](/uploads/befded524f11a4c42ece9fee0611f742/AirflowLogOfFailedzgyComversion.txt)
[SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json](/uploads/3bfd15a13e09d78eb8a23f93cc4036fa/SegyToZgyConversion_Workflow_R3_CI-CD_v2.0.1.postman_collection.json)
@ChrisZhangM9 - Release 0.12Aliaksandr Ramanovich (EPAM)Yan Sushchynski (EPAM)Aliaksandr Ramanovich (EPAM)