Skip to content

Allow opening an existing ZGY file for writing.

Paal Kvamme requested to merge kvamme62/update-existing into master

Allow opening an existing ZGY file for writing with a call to the new IZgyWriter::reopen().

There are several restrictions on using this feature. Some of those might be relaxed in the future.

  • The file must have the latest version. Currently not enforced.

  • size, bricksize, datatype, and datarange are not allowed to change.

  • filename, iocontext, compressor, and lodcompressor are not stored in the file itself. So the caller needs to re-specify them.

  • ilstart, ilinc, xlstart, xlinc, zstart, zinc, zunit, hunit, corners can all be changed but note that this can only be done in the actual call to reopen().

  • The operation is expensive, even when the only change is to update some metadata. The current implementation will delete and re-create the file behind the scenes.

  • The file to be opened cannot contain any bulk data. The only exception is that the entire file may have been initialized to some arbitrary constant.

  • If the file to be updated is stored on a cloud back-end then it must have been created by OpenZGY and written directly to the cloud. The reason: Even though the ZGY-Public and OpenZGY file formats are identical, the data on the cloud can be split into multiple chunks which are logically treated as a single byte stream. The choice of where to split differs between OpenZGY, ZGY-Public, and uploads done by sdutil. This restriction is currently not active but it will become active when a better implementation (removing most of the other rules) has been coded.

  • If the file does contain bulk data (which currently isn't allowed) then it is strongly advised to not change from compressed to uncompressed or vice versa. Even though this might not be checked. If you ignore this advice then he new compression mode will only apply to subsequent writes.

  • If the file does not contain bulk data but is flagged as v4 anyway, it is ok to reopen it as uncompressed. In that case it is unspecified whether the version is changed back from 4 to 3 (which is the current behavior) or not.

  • If the file does contain bulk data (which currently isn't allowed) then it is strongly advised to not update any already existing data. Including overwrite due to a read/modify/write cycle. With a cloud backend this could waste storage and might also be refused at runtime.

  • A file that has never been written to may end up as a file that has been explicitly written as the default (usually zero) value. The OpenZGY api hides that distinction anyway. So it shouldn't matter.

  • The operation can leave the file unusable or even deleted if some error occurs. Since we are creating a brand new file it would be possible to create this with a new name and only rename it and discard the original once we know that the new file is good. But, (a) rename on the cloud is difficult and (b) once this code gets implemented properly that trick isn't possible anyway.

This new feature is available in OpenZGY/C++ and the Python wrapper. Not in the pure Python version.

Merge request reports