api.h 49.2 KB
Newer Older
1
// Copyright 2017-2021, Schlumberger
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#pragma once

#include <cstdint>
#include <vector>
#include <array>
#include <ostream>
#include <iostream>
#include <functional>
#include <memory>
#include <ostream>
#include <iostream>
#include <string>
27
#include <mutex>
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

#include "declspec.h"

#ifdef _MSC_VER
#pragma warning(push)
// The warning is due to me using pure virtual classes as "real" interfaces,
// having a similar inheritance scheme for interfaces and implementation.
// I believe this is safe as long as the interfaces are truly pure.
#pragma warning(disable:4250) // inherits via dominance
#endif

namespace OpenZGY {
  /**
   * \mainpage
   *
   * The %OpenZGY C++ API allows read/write access to files
   * stored in the ZGY format. The main part of the API is here:
   *
   * \li IZgyReader and its IZgyMeta base class.
   * \li IZgyWriter and its ZgyWriterArgs argument package.
   * \li IZgyUtils for anything not read or write.
   * \li \ref exceptions
   *  that you might want to catch.
   * \if SSTORE
   * \li SeismicStoreIOContext for cloud credentials.
   * \endif
   * \li ProgressWithDots example of progress reporting.
   * \li \ref Example Example application.
   *
57
58
59
60
61
   * If you are reading this document from doxygen/native/apidoc.pdf
   * in the source tree then please see doxygen/README.md for an
   * explanation of why the documentation produced by the build might
   * be better.
   *
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
   * \if IMPL
   * If you are viewing the full Doxygen documentation then this
   * covers both the API and most of the implementation. So if you
   * look at the list of classes and methods this might seem a bit
   * daunting. All you really need to use the API should be in the
   * above list. Excluding trivial structs that will be cross
   * referenced as needed. So you don't need to go looking for them.
   * Of course, if you want to work om %OpenZGY itself then you
   * probably need everything.
   *
   * See also the following related pages:
   *
   * \li \ref physicalformat
   * \li \ref implementation
   * \li \ref lowres
   * \li \ref migration
   *
   * \endif
   *
   * \internal
   *
   * The public version of the documentation will include both the
   * IZgyReader and the ZgyReader class, with identical descriptions.
   * Same for IZgyWriter et cetera. I don't want ZgyReader and friends
   * to show up but the documentation is attached to those classes and
   * the interface just has a copydoc. This is done deliberately to
   * keep the documentation close to the inplementation so it is easy
   * to keep in sync.
   *
   * Doxygen's cond statement doesn't do what I want. That will exclude
   * the docmentation completely. Whay I want is to keep the information
   * for use by \copydoc but ignore it for any other purpose.
   *
   * \endinternal
   *
   * \namespace OpenZGY
   *
   * \brief The entire public API is in this namespace.
   *
   * The main interface is IZgyReader and IZgyWriter.
   *
   * \namespace ::OpenZGY::Errors
   *
   * \brief Exceptions that can be thrown by OpenZGY.
   *
   * \namespace ::OpenZGY::Impl
   *
   * \brief Implementation of the abstract interfaces in OpenZGY
   *
   * \namespace ::OpenZGY::Formatters
   *
   * \brief operator<< for readable output of enums etc.
   *
   * \if IMPL
   * \namespace InternalZGY   *
   * \brief Implementation not visible to clients.
   * \endif
   *
   * \page Example
   * \include example.h
   */
}

namespace Test {
  class ZgyWriterMock;
}

namespace OpenZGY {
#if 0
}
#endif

class IOContext;
class IZgyReader;

namespace Impl {
138
139
  class ZgyMeta;
  class ZgyReader;
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
  class ZgyWriter;
  class EnumMapper;
}

/** \file api.h
 *
 * \brief IZgyReader, IZgyWriter, and other user visible classes.
 *
 * This file contains the public OpenZGY API.
 *
 * The API is modeled roughly after the API exposed by the Python wrapper
 * around the existing C++ ZGY-Public API. This is probably just as good
 * a starting point than anything else. And it makes testing simpler for
 * those tests that compare the old and new behavior.
 *
 * OpenZGY::IZgyMeta
 *
 * \li Has a private reference to a impl.meta.ZgyInternalMeta instance.
 * \li Contains a large number of properties exposing meta data,
 *     most of which will present information from the ZgyInternalMeta
 *     in a way that is simpler to use and doesn't depend on the
 *     file version.
 * \li End users will access methods from this class. Most likely
 *     via the derived ZgyReader and ZgyWriter classes. The end users
 *     might not care that there is a separate base class.
 *
 * OpenZGY::ZgyMetaAndTools extends IZgyMeta
 *
 * \li Add coordinate conversion routines to a ZgyMeta instance.
 *
 * OpenZGY::ZgyReader extends ZgyMetaAndTools with read, readconst, close
 * OpenZGY::ZgyWriter extends ZgyMetaAndTools with write, writeconst, finalize, close
 *
 * \li Has a private reference to a impl.meta.ZgyInternalBulk instance.
 * \li Add bulk read and write functionality, forwarding the requests
 *     to the internal bulk instance.
 * \li These classes with their own and inherited methods and properties
 *     comprise the public OpenZGY API.
 * \li Currently the ZgyWriter does not expose the bulk read() function
 *     but it does allow accessing all the metadata. Allowing read()
 *     might be added later if it appears to be useful.
 *     In practice this just means to let ZgyWriter inherit ZgyReader.
 *
 * Note that enums and exceptions are problematic when it comes to
 * encapsulation. Enums might be:
 *
 * \li    Only visible in the public api. Often end up being mapped
 *        to a corresponding internal enum. No problem, but the mapping
 *        can be a bit tedious to maintain.
 *
 * \li    Only visible internally. Possibly representing integer values
 *        written to file. Which makes them an implementation detail,
 *        which must absolutely not be exposed publically.
 *
 * \li    Defined by the public api but passed unchanged from the api
 *        layer to the implementation layer. So there will be an
 *        include in some impl/xxx.h file to a part of the public API.
 *        This is frowned upon.
 *
 * \li    Defined by the internal api but may need to be used from
 *        applications, i.e. also visible in the public api. There will
 *        be an include of e.g. "impl/ugly.h" from one of the public
 *        header files and/or application code.
 *        This is strongly discouraged except for testing.
 *
 * Exceptions have similar issues. It is possible to catch all exceptions
 * raised in the impl layer and convert those to exceptions owned by the
 * api layer. But re-throwing an exception makes debuggig harder.
 */

/**
 * \brief Sample data type used in the public API.
 *
 * Corresponds to RawDataType used in the ZGY file format.
 * Data may be read and written using this type or 32-bit float.
215
216
 *
 * Thread safety: Enums do not have race conditions.
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
 */
enum class OPENZGY_API SampleDataType
{
  unknown = 1000,
  int8    = 1001,
  int16   = 1002,
  float32 = 1003,               // Note, cannot call it just "float".
};

namespace Formatters {
  extern OPENZGY_API std::ostream& operator<<(std::ostream& os, SampleDataType value);
}

/**
 * \brief Horizontal or vertical dimension as used in the public API.
 *
 * Horizontal dimension may be length or arc angle, although most
 * applications only support length. Vertical dimension may be time or
 * length. Vertical length is of course the same as depth. Arguably
 * there should have been separate enums for horizontal and vertical
 * dimension since the allowed values differ.
238
239
 *
 * Thread safety: Enums do not have race conditions.
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
 */
enum class OPENZGY_API UnitDimension
{
  unknown  = 2000,
  time     = 2001,
  length   = 2002,
  arcangle = 2003,
};

namespace Formatters {
  extern OPENZGY_API std::ostream& operator<<(std::ostream& os, UnitDimension value);
}

/**
 * \brief Possible algorithms to generate LOD bricks.
 *
 * We might trim this list later to what is actually in use.
 * The "classic" ZGY only uses the first two.
 *
 * Maps to LodAlgorithm in the implementation layer.
260
261
 *
 * Thread safety: Enums do not have race conditions.
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
 */
enum class OPENZGY_API DecimationType
{
    LowPass = 100,    ///< \brief Lowpass Z / decimate XY.
    WeightedAverage,  ///< \brief Weighted averaging (depends on global stats).
    Average,          ///< \brief Simple averaging.
    Median,           ///< \brief Somewhat more expensive averaging.
    Minimum,          ///< \brief Minimum value.
    Maximum,          ///< \brief Maximum value.
    MinMax,           ///< \brief Checkerboard of minimum and maximum values.
    Decimate,         ///< \brief Simple decimation, use first sample.
    DecimateSkipNaN,  ///< \brief Use first sample that is not NaN.
    DecimateRandom,   ///< \brief Random decimation using a fixed seed.
    AllZero,          ///< \brief Just fill the LOD brick with zeroes.
    WhiteNoise,       ///< \brief Fill with white noise, hope nobody notices.
    MostFrequent,     ///< \brief The value that occurs most frequently.
    MostFrequentNon0, ///< \brief The non-zero value that occurs most frequently.
    AverageNon0,      ///< \brief Average value, but treat 0 as NaN.
};

namespace Formatters {
  extern OPENZGY_API std::ostream& operator<<(std::ostream& os, DecimationType value);
}

/**
 * \brief Statistics of all sample values on the file.
288
289
290
291
292
 *
 * \details Thread safety:
 * Modification may lead to a data race. This should not be an issue,
 * because instances are only meant to be modified when created or
 * copied or assigned prior to being made available to others.
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
 */
class OPENZGY_API SampleStatistics
{
public:
  std::int64_t cnt;  /**< \brief Number of added samples. */
  double sum;        /**< \brief Sum of added samples. */
  double ssq;        /**< \brief Sum-of-squares of added samples. */
  double min;        /**< \brief Minimum added sample value. */
  double max;        /**< \brief Maximum added sample value. */
  SampleStatistics()
    : cnt(0), sum(0), ssq(0), min(0), max(0)
  {
  }

  /** \brief Create a new instance with all contents filled in. */
  SampleStatistics(std::int64_t cnt_in,
                 double sum_in, double ssq_in,
                 double min_in, double max_in)
    : cnt(cnt_in), sum(sum_in), ssq(ssq_in), min(min_in), max(max_in)
  {
  }
};

/**
 * \brief Histogram of all sample values on the file.
 *
 * \internal
 * KEEP THIS TEXT IN SYNC WITH InternalZGY::HistogramData
 * \endinternal
 *
 * The histogram is described by the fixed total number of bins, the
 * center value of the samples in the first bin, and the center value
 * of the samples in the last bin.
 *
 * The width of each bin is given by (max - min) / (nbins - 1).
 * Bin 0 holds samples with values min_ +/-binwidth/2.
 *
 * This means that the total range of samples that can be represented in the
 * histogram is actually min-binwidth/2 to max+binwidth/2, which is slightly
 * larger than just min..max _unless_ each of the bins can only hold a single
 * value (e.g. data converted from 8-bit storage, in 256 bins).
 *
 * This definition has some subtle effects, best illustrated by a few examples.
 *
 * If the source data is ui8_t, the logical range is 0..255. Assume nbins=256.
 * This gives binwidth=1, and the true range of the histogram -0.5..+255.5.
 * But since the input is integral, the actual range is just 0..255 inclusive.
 * Try to fill the histogram with evenly distrubuted random data and you end
 * up with each bin having roughly the same number of elements.
 *
 * Now consider ui16_t, range 0..65535 and nbins is still 256. This gives
 * binwidth=257, not 256. The true range of the histogram is -128.5..+65663.5.
 * Try to fill the histogram with evenly distrubuted random data and you end
 * up with the first and the last bin having approximately half as many
 * elements as all the others. This is not really a problem, but may seem
 * a bit surprising.
349
350
351
352
 *
 * The range should have the zero-centric property. Zero, if present in the
 * range, should map to the center of a bin. Otherwise the histogram close to
 * zero might look odd.
353
354
355
356
357
 *
 * Thread safety:
 * Modification may lead to a data race. This should not be an issue,
 * because instances are only meant to be modified when created or
 * copied or assigned prior to being made available to others.
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
 */
class OPENZGY_API SampleHistogram
{
public:
  std::int64_t samplecount; /**< \brief Sum of counts of all bins */
  double minvalue;          /**< \brief Center value of data in first bin */
  double maxvalue;          /**< \brief Center value of data in last bin */
  std::vector<std::int64_t> bins; /**< \brief array of counts by bin */
  SampleHistogram()
    : samplecount(0), minvalue(0), maxvalue(0), bins()
  {
  }

  /** \brief Create a new instance with all contents filled in. */
  SampleHistogram(std::int64_t samplecount_in,
                double minvalue_in, double maxvalue_in,
                const std::vector<std::int64_t>& bins_in)
    : samplecount(samplecount_in)
    , minvalue(minvalue_in)
    , maxvalue(maxvalue_in)
    , bins(bins_in)
  {
  }
};

383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
/**
 * \brief Return value from filestats().
 * \details
 *
 * Meta information about the file that might be reported to a curious
 * end user. Such as the compression factor that was achieved. Or whether
 * this file was written by a legacy application that chose to store
 * alpha tiles in the file. The information is not needed to consume
 * the data stored in the file.
 *
 * This is a concrete class, not an interface. This is done to make
 * the class a POD and ensure it is copyable. Note this means you
 * should not assign the filestats() result to a referece.
 *
 * Note that the reported compression is about file sizes, not the
 * requested signal to noise factor. The latter would need to be
 * saved explicitly in the file. Currently it isn't.
 *
 * Normally the only wasted size in a ZGY file is the padding between
 * the header area and the first brick. For a file with aplha tiles
 * (cannot currently be cretated by OpenZGY) there might be more data
 * lost due to alignment. And files written to cloud storage in a way
 * that updates the same brick more than once may end up with holes.
 *
 * In the statistics, the padding after the header area is captured
 * in padding_size and all other holes are captured in wasted_size.
 * Caveat: Leftover space after the end of a compressed block will
 * currently not be detected.
 */
class OPENZGY_API FileStatistics
{
 // The code that fills in the contents do so by accessing the data fields
 // directly. There isn't much point in the additional boilerplace code
 // to implement write accessors.
 friend class Impl::ZgyMeta;
 friend class Impl::ZgyReader;
 friend class Impl::ZgyWriter;

private:
  std::int64_t _file_version;
  std::int64_t _file_size;
  std::int64_t _header_size;
  //std::int64_t _padding_size;
  //std::int64_t _wasted_size;
  std::int64_t _alpha_normal_count;
  std::int64_t _alpha_normal_size_per_entry;
  std::int64_t _alpha_compressed_count;
  std::int64_t _alpha_compressed_size;
  std::int64_t _alpha_missing_count;
  std::int64_t _alpha_constant_count;
  std::int64_t _brick_normal_count;
  std::int64_t _brick_normal_size_per_entry;
  std::int64_t _brick_compressed_count;
  std::int64_t _brick_compressed_size;
  std::int64_t _brick_missing_count;
  std::int64_t _brick_constant_count;
  // Derived information
  std::int64_t _used_size;
  std::int64_t _used_if_uncompressed;
  double _compression_factor;
  bool _is_compressed;

public:
FileStatistics()
    : _file_version(0)
    , _file_size(0)
    , _header_size(0)
    //, _padding_size(0)
    //, _wasted_size(0)
    , _alpha_normal_count(0)
    , _alpha_normal_size_per_entry(0)
    , _alpha_compressed_count(0)
    , _alpha_compressed_size(0)
    , _alpha_missing_count(0)
    , _alpha_constant_count(0)
    , _brick_normal_count(0)
    , _brick_normal_size_per_entry(0)
    , _brick_compressed_count(0)
    , _brick_compressed_size(0)
    , _brick_missing_count(0)
    , _brick_constant_count(0)
    , _used_size(0)
    , _used_if_uncompressed(0)
    , _compression_factor(1.0)
    , _is_compressed(false)
  {
  }

  /// Version number from the main file header.
  std::int64_t fileVersion() const { return _file_version; }
  /// Total size of the file on disk or cloud.
  std::int64_t fileSize() const { return _file_size; }
  /// Size of all headers.
  std::int64_t headerSize() const { return _header_size; }
  // Wasted due to first brick alignment.
  //std::int64_t paddingSize() const { return _padding_size; }
  // Wasted due to other reasons.
  //std::int64_t wastedSize() const { return _wasted_size; }
  /// Number of uncompressed tiles.
  std::int64_t alphaNormalCount() const { return _alpha_normal_count; }
  /// Size used by one uncompressed tile.
  std::int64_t alphaNormalSizePerEntry() const { return _alpha_normal_size_per_entry; }
  /// Number of compressed tiles.
  std::int64_t alphaCompressedCount() const { return _alpha_compressed_count; }
  /// Total size used by compressed tiles.
  std::int64_t alphaCcompressedSize() const { return _alpha_compressed_size; }
  /// Number of compressed tiles.
  std::int64_t alphaMissingCount() const { return _alpha_missing_count; }
  /// Number of constant value tiles.
  std::int64_t alphaConstantCount() const { return _alpha_constant_count; }
  /// Number of uncompressed bricks.
  std::int64_t brickNormalCount() const { return _brick_normal_count; }
  /// Size used by one uncompressed brick.
  std::int64_t brickNormalSizePerEntry() const { return _brick_normal_size_per_entry; }
  /// Number of compressed bricks.
  std::int64_t brickCompressedCount() const { return _brick_compressed_count; }
  /// Total size used by compressed bricks.
  std::int64_t brickCompressedSize() const { return _brick_compressed_size; }
  /// Number of compressed bricks.
  std::int64_t brickMissingCount() const { return _brick_missing_count; }
  /// Number of constant value bricks.
  std::int64_t brickConstantCount() const { return _brick_constant_count; }
  // Derived information
  /**
   * \brief Space used by headers and data bricks.
   * \details
   * File size not including padding and holes, combining all LOD
   * levels and the main headers. The padding between the header area
   * and the first brick will not be included. Nor will holes between
   * uncompressed bricks contribute. Currently any holes between
   * compressed bricks are not detected which means that they will be
   * counted as used. This can be derived from the other information.
   */
  std::int64_t usedSize() const { return _used_size; }
  /**
   * \brief Space needed if the file is/was uncompressed.
   * \details
   * As used_size if the file is/was uncompressed. This can be derived
   * from the other information.
   */
  std::int64_t usedIfUncompressed() const { return _used_if_uncompressed; }
  /**
   * \brief Measure how successful the compression was.
   * \details
   * Estimate the relative size of this possibly compressed file
   * compared to the same file if uncompressed. Will be 1.0 if file is
   * already uncompressed but a value of 1.0 doesn't technically imply
   * that the file is not compressed. Padding is ignored so the result
   * will not match precisely what you get by uncompressing the file
   * and storing it on disk. Also not taken into account is that the
   * padding after the header area might differ between the compressed
   * and uncompressed formats.
   */
  double compressionFactor() const { return _compression_factor; }
  /**
   * True if at least one brick is flagged as compressed, even in the
   * unlikely case where the compression didn't actually reduce the
   * file size. This can be derived from the other information.
   */
  bool isCompressed() const { return _is_compressed; }
  /**
   * For debugging. Output most of the information to the supplied ostream.
   */
  void dump(std::ostream& out, const std::string& prefix = "") const {
    out << prefix << "ZGY version " << _file_version
        << " file compressed to "
        << int(100.0 * _compression_factor) << "% of original\n"
        << prefix << "Size:  "
        << _file_size << " bytes of which "
        << _header_size << " are in headers and "
        << _file_size - _used_size << " wasted\n"
        << prefix << "Alpha: "
        << _alpha_missing_count    << " missing, "
        << _alpha_constant_count   << " constant, "
        << _alpha_normal_count     << " normal ("
        << _alpha_normal_count * _alpha_normal_size_per_entry << " bytes), "
        << _alpha_compressed_count << " compressed ("
        << _alpha_compressed_size  << " bytes)\n"
        << prefix << "Brick: "
        << _brick_missing_count    << " missing, "
        << _brick_constant_count   << " constant, "
        << _brick_normal_count     << " normal ("
        << _brick_normal_count * _brick_normal_size_per_entry << " bytes), "
        << _brick_compressed_count << " compressed ("
        << _brick_compressed_size  << " bytes)\n";
  }
};

571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
/**
 * \brief Argument package for creating a ZGY file.
 *
 * This is a short lived helper class for passing arguments to the functions
 * that create a ZGY file. Needed because there are a lot of arguments
 * and C++ doesn't allow keyword arguments. Do NOT use this class
 * for holding on to the information.
 *
 * Information not set explicitly will be defaulted. The following
 * two statements give the same result:
 * \code
 *   ZgyWriterArgs default_args = ZgyWriterArgs();
 *
 *   ZgyWriterArgs default_args = ZgyWriterArgs()
 *     .filename("")
 *     .iocontext(nullptr)
 *     .compressor(nullptr)
 *     .lodcompressor(nullptr)
 *     .size(0, 0, 0)
 *     .bricksize(64, 64, 64)
 *     .datatype(SampleDataType::float32)
 *     .datarange(0, -1)
 *     .zunit(UnitDimension::unknown, "", 1.0)
 *     .hunit(UnitDimension::unknown, "", 1.0)
 *     .ilstart(0)
 *     .ilinc(0)
 *     .xlstart(0)
 *     .xlinc(0)
 *     .zstart(0)
 *     .zinc(0)
 *     .corners(ZgyWriterArgs::corners_t{0,0,0,0,0,0,0,0});
 * \endcode
603
604
605
606
607
 *
 * Thread safety:
 * Modification may lead to a data race. This should not be an issue,
 * because instances are only meant to be modified when created or
 * copied or assigned prior to being made available to others.
608
609
610
611
612
613
614
615
616
617
618
619
620
621
 */
class OPENZGY_API ZgyWriterArgs
{
public:
  typedef double float64_t;
  typedef std::array<std::array<float64_t,2>,4> corners_t;
  // Type aliases duplicated from the InternalZGY layer.
  typedef std::pair<std::shared_ptr<const void>, std::int64_t> rawdata_t;
  typedef std::function<rawdata_t(const rawdata_t&,const std::array<int64_t,3>&)> compressor_t;

private:
  friend class Impl::ZgyWriter; // Processing the high level parts.
  friend class ::Test::ZgyWriterMock;
  friend class Impl::EnumMapper; // Mapping the rest to the internal struct.
622
  friend class IZgyWriter; // For IZgyWriter::reopen()
623
624
625
626
627
628
629
630
631
632
633
634
635
636
  std::string _filename;
  const IOContext *_iocontext; // TODO-Worry: Sure instance is short lived?
  compressor_t _compressor;
  compressor_t _lodcompressor;
  std::array<std::int64_t,3> _size;
  std::array<std::int64_t,3> _bricksize;
  SampleDataType _datatype;
  std::array<float,2> _datarange;
  UnitDimension _zunitdim, _hunitdim;
  std::string _zunitname, _hunitname;
  double _zunitfactor, _hunitfactor;
  float _zstart, _zinc;
  std::array<float,2> _annotstart, _annotinc;
  corners_t _corners;
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
  // Keeps track of what information has been changed from the default.
  // Currently metafrom() will set all to true. That might change.
  // The information is only used by merge(). Transient information
  // such as iocontext is ignored.
  bool _have_size;
  bool _have_bricksize;
  bool _have_datatype;
  bool _have_datarange;
  bool _have_zunit;
  bool _have_hunit;
  bool _have_ilstart;
  bool _have_ilinc;
  bool _have_xlstart;
  bool _have_xlinc;
  bool _have_zstart;
  bool _have_zinc;
  bool _have_corners;
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675

public:
  ZgyWriterArgs()
    : _filename("")
    , _iocontext(nullptr)
    , _compressor()
    , _lodcompressor()
    , _size{0,0,0}
    , _bricksize{64,64,64}
    , _datatype(SampleDataType::float32)
    , _datarange{0, -1}
    , _zunitdim(UnitDimension::unknown)
    , _hunitdim(UnitDimension::unknown)
    , _zunitname("")
    , _hunitname("")
    , _zunitfactor(1.0)
    , _hunitfactor(1.0)
    , _zstart(0)
    , _zinc(0)
    , _annotstart{0,0}
    , _annotinc{0,0}
    , _corners{0,0,0,0,0,0,0,0}
676
677
678
679
680
681
682
683
684
685
686
687
688
    , _have_size(false)
    , _have_bricksize(false)
    , _have_datatype(false)
    , _have_datarange(false)
    , _have_zunit(false)
    , _have_hunit(false)
    , _have_ilstart(false)
    , _have_ilinc(false)
    , _have_xlstart(false)
    , _have_xlinc(false)
    , _have_zstart(false)
    , _have_zinc(false)
    , _have_corners(false)
689
690
691
692
693
694
  {
  }

  /**
   * \brief Output in human readable form for debugging.
   */
695
  void dump(std::ostream& out) const
696
697
698
699
700
701
  {
    out << "ZgyWriterArgs\n"
        << "  filename:    \"" << _filename << "\"\n"
        << "  iocontext:   " << (_iocontext ? "*" : "(null)") << "\n"
        << "  compressor:  " << (_compressor ? "*" : "(null)") << "\n"
        << "  lodcompress: " << (_lodcompressor ? "*" : "(null)") << "\n"
702
703
704
705
706
707
708
709
710
711
712
713
714
        << "  " << (_have_size?"*":"") << "size:        (" << _size[0] << "," << _size[1] << "," << _size[2] << ")\n"
        << "  " << (_have_bricksize?"*":"") << "bricksize:   (" << _bricksize[0] << "," << _bricksize[1] << "," << _bricksize[2] << ")\n"
        << "  " << (_have_datatype?"*":"") << "datatype:    " << int(_datatype) << "\n"
        << "  " << (_have_datarange?"*":"") << "datarange:   " << _datarange[0] << " to " << _datarange[1] << "\n"
        << "  " << (_have_zunit?"*":"") << "zunit:       " << int(_zunitdim) << " \"" << _zunitname << "\" " << _zunitfactor << "\n"
        << "  " << (_have_hunit?"*":"") << "hunit:       " << int(_hunitdim) << " \"" << _hunitname << "\" " << _hunitfactor << "\n"
        << "  " << (_have_ilstart||_have_ilinc?"*":"") << "ilstart/inc: " << _annotstart[0] << " / " << _annotinc[0] << "\n"
        << "  " << (_have_xlstart||_have_xlinc?"*":"") << "xlstart/inc: " << _annotstart[1] << " / " << _annotinc[1] << "\n"
        << "  " << (_have_zstart||_have_zinc?"*":"") << "zstart/inc:  "  << _zstart << " / " << _zinc << "\n"
        << "  " << (_have_corners?"*":"") << "corner0:     " << _corners[0][0] << ", " << _corners[0][1] << "\n"
        << "  " << (_have_corners?"*":"") << "corner1:     " << _corners[1][0] << ", " << _corners[1][1] << "\n"
        << "  " << (_have_corners?"*":"") << "corner2:     " << _corners[2][0] << ", " << _corners[2][1] << "\n"
        << "  " << (_have_corners?"*":"") << "corner3:     " << _corners[3][0] << ", " << _corners[3][1] << "\n";
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
      }

  /**
   * \brief Set file to open.
   * \if SSTORE
   * \details If starting with sd:// this opens a file on seismic store.
   * \endif
   */
  ZgyWriterArgs& filename(const std::string& value) { _filename = value; return *this; }
  /**
   * \brief Set credentials and other configuration.
   * \details This depends on the back-end.
   * For local files you normally don't need to pass anything here.
   */
  ZgyWriterArgs& iocontext(const IOContext *value) { _iocontext = value; return *this; }
  /**
   * \brief Set functor for compressing full resolution data.
   */
  ZgyWriterArgs& compressor(const compressor_t& value) { _compressor = value; return *this; }
  /**
   * \brief Set functor for compressing full resolution data.
   * \details This overload uses a factory to look up the compressor.
   */
  ZgyWriterArgs& compressor(const std::string& name, const std::vector<std::string>& args);
  /**
   * \brief Set functor for compressing full resolution data.
   * \details This overload uses a factory to look up the ZGY compressor.
   * It is just a convenience that is shorter to type.
   */
  ZgyWriterArgs& zfp_compressor(float snr);
  /**
   * \brief Set functor for compressing low resolution data.
   */
  ZgyWriterArgs& lodcompressor(const compressor_t& value) { _lodcompressor = value; return *this; }
  /**
   * \brief Set functor for compressing low resolution data.
   * \details This overload uses a factory to look up the compressor.
   */
  ZgyWriterArgs& lodcompressor(const std::string& name, const std::vector<std::string>& args);
  /**
   * \brief Set functor for compressing low resolution data.
   * \details This overload uses a factory to look up the ZGY compressor.
   * It is just a convenience that is shorter to type.
   */
  ZgyWriterArgs& zfp_lodcompressor(float snr);
  /**
   * \brief Set size of the survey.
   * \param ni number of inlines (slowest varying index).
   * \param nj number of crosslines.
   * \param nk number of samples per trace (fastest).
   */
766
  ZgyWriterArgs& size(std::int64_t ni, std::int64_t nj, std::int64_t nk) { _size[0]=ni; _size[1]=nj; _size[2]=nk; _have_size = true; return *this; }
767
768
769
770
  /**
   * \brief Set size of one brick.
   * \details Almost always (64,64,64). Change at your own peril.
   */
771
  ZgyWriterArgs& bricksize(std::int64_t ni, std::int64_t nj, std::int64_t nk) { _bricksize[0]=ni; _bricksize[1]=nj; _bricksize[2]=nk; _have_bricksize = true; return *this; }
772
773
774
  /**
   * \brief Set type of samples in each brick.
   */
775
  ZgyWriterArgs& datatype(SampleDataType value) { _datatype = value; _have_datatype = true; return *this; }
776
777
778
779
780
781
  /**
   * \brief Set scaling factors.
   * \details For integral storage this specifies the two floating
   * point numbers that correspond to the lowest and highest
   * representable integer value. So for int8 files, -128 will be
   * converted to "lo" and +127 will be converted to "hi".
782
783
784
   *
   * ZGY files require that min<max. This is not enforced here,
   * but is checked when the file is actually created.
785
   */
786
ZgyWriterArgs& datarange(float lo, float hi) { _datarange[0] = lo; _datarange[1] = hi; _have_datarange = true; return *this; }
787
788
789
790
791
792
793
794
795
796
  /**
   * \brief Set vertical unit.
   * \param dimension time or depth (a.k.a. length).
   * \param name for annotation only.
   * \param factor multiply by this factor to convert from storage units to SI units.
   */
  ZgyWriterArgs& zunit(UnitDimension dimension, const std::string& name, double factor) {
    _zunitdim = dimension;
    _zunitname = name;
    _zunitfactor = factor;
797
    _have_zunit = true;
798
799
800
801
802
803
804
805
806
807
808
809
    return *this;
  }
  /**
   * \brief Set horizontal unit.
   * \param dimension cartesian length or (unsupported) polat coordinates,
   * \param name for annotation only.
   * \param factor multiply by this factor to convert from storage units to SI units.
   */
  ZgyWriterArgs& hunit(UnitDimension dimension, const std::string& name, double factor) {
    _hunitdim = dimension;
    _hunitname = name;
    _hunitfactor = factor;
810
    _have_hunit = true;
811
812
813
814
815
816
    return *this;
  }
  /// \brief Set first (ordinal 0) inline number.
  /// \details For maximum portability the inline and crossline start
  /// and increment should be integral numbers. Some applications
  /// might choose to convert them to int.
817
  ZgyWriterArgs& ilstart(float value) { _annotstart[0] = value; _have_ilstart = true; return *this; }
818
819
  /// \brief Set inline number increment between two adjacent ordinal values.
  /// \copydetails ilstart
820
  ZgyWriterArgs& ilinc(float value) { _annotinc[0] = value; _have_ilinc = true; return *this; }
821
822
  /// \brief Set first (ordinal 0) crossline number.
  /// \copydetails ilstart
823
  ZgyWriterArgs& xlstart(float value) { _annotstart[1] = value; _have_xlstart = true; return *this; }
824
825
  /// \brief Set crossline number increment between two adjacent ordinal values.
  /// \copydetails ilstart
826
  ZgyWriterArgs& xlinc(float value) { _annotinc[1] = value; _have_xlinc = true; return *this; }
827
828
  /// \brief Set first time/depth.
  /// \details Vertical annotation is generally safe to have non-integral.
829
  ZgyWriterArgs& zstart(float value) { _zstart = value; _have_zstart = true; return *this; }
830
831
  /// \brief Set increment (distance between samples) in vertical direction.
  /// \copydetails zstart
832
  ZgyWriterArgs& zinc(float value) { _zinc = value; _have_zinc = true; return *this; }
833
834
835
  /// \brief Set survey corner points in world coordinates.
  /// \details The corners are ordered origin, last inline (i.e. i=last, j=0),
  /// last crossline, diagonal.
836
  ZgyWriterArgs& corners(const corners_t& value) { _corners = value; _have_corners = true; return *this; }
837
838
839
840
841
842
843
844
845
846
  /**
   * \brief Copy metadata from existing file.
   *
   * Set most of the metadata from an open file descriptor.
   * Typically used when the file to be created is to be computed from
   * an existing file. Typically this will be called first so the
   * settings don't inadverently shadow something set explicitly.
   */
  ZgyWriterArgs& metafrom(const std::shared_ptr<OpenZGY::IZgyReader>&);

847
848
849
850
851
852
853
854
855
  /**
   * \brief Copy metadata from another ZgyWriterArgs.
   *
   * Copy only those settings that have been explicitly changed in the
   * supplied "other" ZgyWriterArgs into *this. If other.metafrom()
   * has been called it is unspecified what gets copied.
   */
  ZgyWriterArgs& merge(const ZgyWriterArgs&);

856
857
858
859
860
861
  // TODO-Low: Add accessors as well. But these are only for internal
  // use so the ugliness of accessing data members isn't that bad.
};

/**
 * \brief Base class of IZgyReader and IZgyWriter.
862
 * \details Thread safety: Interfaces do not have race conditions.
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
 */
class OPENZGY_API IZgyMeta
{
public:
  typedef std::int8_t int8_t;
  typedef std::int16_t int16_t;
  typedef std::int32_t int32_t;
  typedef std::int64_t int64_t;
  typedef float float32_t;
  typedef double float64_t;
  typedef std::array<int64_t,3> size3i_t;
  typedef std::array<std::array<float64_t,2>,4> corners_t;
  // Type aliases duplicated from the InternalZGY layer.
  typedef std::pair<std::shared_ptr<const void>, std::int64_t> rawdata_t;
  typedef std::function<rawdata_t(const rawdata_t&,const std::array<int64_t,3>&)> compressor_t;

public:
  virtual ~IZgyMeta();
  virtual size3i_t       size()           const = 0; /**< \brief Size in inline, crossline, vertical directions. */
  virtual SampleDataType datatype()       const = 0; /**< \brief Type of samples in each brick. */
  virtual std::array<float32_t,2> datarange() const = 0; /**< \brief Used for float to int scaling. */
884
  virtual std::array<float32_t,2> raw_datarange() const = 0; /**< \brief datarange before adjustment. */
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
  virtual UnitDimension  zunitdim()       const = 0; /**< \brief Vertical dimension. */
  virtual UnitDimension  hunitdim()       const = 0; /**< \brief Horizontal dimension. */
  virtual std::string    zunitname()      const = 0; /**< \brief For annotation only. Use hunitfactor, not the name, to convert to or from SI. */
  virtual std::string    hunitname()      const = 0; /**< \brief For annotation only. Use hunitfactor, not the name, to convert to or from SI. */
  virtual float64_t      zunitfactor()    const = 0; /**< \brief Multiply by this factor to convert from storage units to SI units. */
  virtual float64_t      hunitfactor()    const = 0; /**< \brief Multiply by this factor to convert from storage units to SI units. */
  virtual float32_t      zstart()         const = 0; /**< \brief First time/depth. */
  virtual float32_t      zinc()           const = 0; /**< \brief Increment in vertical direction. */
  virtual std::array<float32_t,2> annotstart() const = 0; /**< \brief First inline, crossline. */
  virtual std::array<float32_t,2> annotinc()   const = 0; /**< \brief Increment in inline, crossline directions. */
  virtual const corners_t& corners()      const = 0; /**< \brief Survey corner points in world coordinates. */
  virtual const corners_t& indexcorners() const = 0; /**< \brief Survey corner points in ordinal (i,j) coordinates. */
  virtual const corners_t& annotcorners() const = 0; /**< \brief Survey corner points in inline, crossline coordinates. */
  virtual size3i_t       bricksize()      const = 0; /**< \brief Size of one brick. Almost always (64,64,64), change at your own peril. */
  virtual std::vector<size3i_t> brickcount() const = 0; /**< \brief Number of bricks at each resolution (LOD) level. */
  virtual int32_t        nlods()          const = 0; /**< \brief Number of resolution (LOD) levels. */
901
902
903
904
905
906
907
  // Only expose the guids we think the application will need.
  // Note that OpenZGY doesn't really support updates,
  // so dataid() and previd() are not very useful.
  //virtual std::string dataid() const = 0;    /**< GUID set on file creation. */
  virtual std::string verid()  const = 0;    /**< GUID set each time the file is changed. */
  //virtual std::string previd() const = 0;    /**< GUID before last change. */

908
909
910
911
912
913
914
915
  // The Python version has meta() as a dict holding all the meta data,
  // this isn't really useful in C++ and just makes it harder to see which
  // metadata is being used. [set_]numthreads is N/A in this accessor.
  // If the code allows enabling multi threading then this would be configured
  // in IOContext or as arguments to the compress plugin.
  //virtual void           meta()           const = 0;
  //virtual int32_t        numthreads()     const = 0;
  //virtual void           set_numthreads(int32_t) = 0;
916
917
918
  virtual void           dump(std::ostream&) const = 0; /**< \brief Output in human readable form for debugging. */
  virtual SampleStatistics statistics()     const = 0; /**< \brief Statistics of all sample values on the file. */
  virtual SampleHistogram  histogram()      const = 0; /**< \brief Histogram of all sample values on the file. */
919
  virtual FileStatistics   filestats()      const = 0; /**< \brief For display purposes only. */
920
921
922
923
};

/**
 * \brief Base class of IZgyReader and IZgyWriter.
924
 * \details Thread safety: Interfaces do not have race conditions.
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
 */
class OPENZGY_API IZgyTools : virtual public IZgyMeta
{
public:
  virtual ~IZgyTools();
  /**
   * \brief General coordinate conversion of an array of points.
   * (NOT IMPLEMENTED YET)
   *
   * \param from: control points in the current coordinate system.
   * \param to:   control points in the desired coordinate system.
   *
   * \details Convert coordinates in place, with the conversion
   * defined by giving the values of 3 arbitrary control points in
   * both the "from" and "to" coordinate system. A common choice of
   * arbitrary points is to use three of the lattice corners.
   * As a convenience the "from" and "to" parameters are declared as
   * corners_t so the caller can pass corners(), annotcorners(), or
   * indexcorners() directly.
   *
   * \internal TODO-Low I haven't implemented the array version yet,
   * only the one that converts a single point.
   */
  virtual void transform(const corners_t& from, const corners_t& to, std::vector<std::array<float64_t,2>>&) const = 0;
  /**
   * \brief General coordinate conversion of a single coordinate pair.
   * \copydetails IZgyTools::transform()
   */
  virtual std::array<float64_t,2> transform1(const corners_t& from, const corners_t& to, const std::array<float64_t,2>&) const = 0;
  virtual std::array<float64_t,2> annotToIndex(const std::array<float64_t,2>&) const = 0; /**< \brief Convert a single coordinate pair. */
  virtual std::array<float64_t,2> annotToWorld(const std::array<float64_t,2>&) const = 0; /**< \brief Convert a single coordinate pair. */
  virtual std::array<float64_t,2> indexToAnnot(const std::array<float64_t,2>&) const = 0; /**< \brief Convert a single coordinate pair. */
  virtual std::array<float64_t,2> indexToWorld(const std::array<float64_t,2>&) const = 0; /**< \brief Convert a single coordinate pair. */
  virtual std::array<float64_t,2> worldToAnnot(const std::array<float64_t,2>&) const = 0; /**< \brief Convert a single coordinate pair. */
  virtual std::array<float64_t,2> worldToIndex(const std::array<float64_t,2>&) const = 0; /**< \brief Convert a single coordinate pair. */
};

/**
 * \brief Main API for reading ZGY files.
 *
 * Obtain a concrete instance by calling the factory method IZgyReader::open().
 * You can then use the instance to read both meta data and bulk data.
 * It is recommended to explicitly close the file when done with it.
 *
 * Note: Yes, I understand that the open() factory method should not have
 * been lexically scoped inside the ostensibly pure %IZgyReader interface.
 * But this reduces the number of classes a library user needs to relate to.
972
973
 *
 * Thread safety: Interfaces do not have race conditions.
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
 */
class OPENZGY_API IZgyReader : virtual public IZgyTools
{
public:
  virtual ~IZgyReader();
  /// \copydoc Impl::ZgyReader::read(const size3i_t&,const size3i_t&,float*,int) const
  virtual void read(const size3i_t& start, const size3i_t& size, float* data, int lod = 0) const = 0;
  /// \copydoc Impl::ZgyReader::read(const size3i_t&,const size3i_t&,std::int16_t*,int) const
  virtual void read(const size3i_t& start, const size3i_t& size, std::int16_t* data, int lod = 0) const = 0;
  /// \copydoc Impl::ZgyReader::read(const size3i_t&,const size3i_t&,std::int8_t*,int) const
  virtual void read(const size3i_t& start, const size3i_t& size, std::int8_t* data, int lod = 0) const = 0;
  /// \copydoc Impl::ZgyReader::readconst()
  virtual std::pair<bool,double> readconst(const size3i_t& start, const size3i_t& size, int lod = 0, bool as_float = true) const = 0;
  /// \copydoc Impl::ZgyReader::close()
  virtual void close() = 0;
  /// \brief Open a ZGY file for reading.
  static std::shared_ptr<IZgyReader> open(const std::string& filename, const IOContext* iocontext = nullptr);
};

/**
 * \brief Main API for creating ZGY files.
 *
 * Obtain a concrete instance by calling the factory method IZgyWriter::open().
 * All meta data is specified in the call to open(), so meta data will appear
 * to be read only. You can use the instance to write bulk data. The file
 * becomes read only once the instance is closed.
 *
For faster browsing, not all history is shown. View entire blame