Merge branch feature/jorgen.lind/python_cleanup_request with refs/heads/master into refs/merge-requests/214/train

Silvian Bensdorp authored Jul 28, 2020 and Jørgen Lind committed Aug 24, 2020

Supports all data types except 'any', and works for 2d and 3d data.
The blocksize and LOD level are currently hard-coded.

This reverts commit eb6e0338.

Jim King authored Aug 20, 2020 and Jørgen Lind committed Aug 20, 2020

remove LRU cache and revert back to a cache similar to the original design, except that we explicitly tell the cache when to drop completed inlines

Fixed that IsOpen() would report the file was still open after it was closed if EnableWriting() had been called.

Added a method to create index entries for chunk data which allocates space in the VDS file and makes it possible to write the data without holding a lock.

Allow NULL data pointer to WriteChunk if we want to write an empty chunk with metadata. Added WriteIndexEntry method to enable async operations. Made BulkDataStore error message thread-local.

Make sure we append persistentID to the URL in a consistent way (which doesn't crash if the base URL is empty).

Reworked changes to allow object store datasets without metadata tags and improve verification of serialized data.

Change HueBulkDataStore to use File class that has thread-safe Read/Write methods. Added IsReadOnly() call.

Make VolumeDataStore destructor virtual so we actually call the derived class's destructor during Close().

Fix missing Commit of changes to SerializedVolumeDataLayout and missing update of FileMetadata on Flush.

Set ProduceStatuses on loading serialized VolumeDataLayout, this can definitly be improved but for now we do it at the same place as VolumeDataStoreIOManager. Return correct CompressionInfo from ReadChunk().

Added basic implementation of WriteSerializedVolumeDataLayout and WriteChunk to VolumeDataStoreVDSFile.