Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Register
  • Sign in
  • O Open ZGY
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 3
    • Issues 3
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe SoftwareOpen Subsurface Data Universe Software
  • Platform
  • Domain Data Management Services
  • Seismic
  • Open ZGY
  • Merge requests
  • !12

Increase chunk size for cloud read.

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Paal Kvamme requested to merge kvamme62/consolidate-algo into master Nov 13, 2020
  • Overview 0
  • Commits 5
  • Pipelines 0
  • Changes 9

Add code to increase performance by reading larger chunks at a time from the cloud.

When the application requests a region that covers more then one brick, the naive approach is to loop and read one brick at a time. The logic added in this merge request will consolidate reads that happen to be adjacent or nearly adjacent.

This will often be the case. Many applications will want to read full traces. And when creating a ZGY file it is recommended to order the bricks with the vertical direction varying fastest.

If the intended use of a ZGY file is to not read traces then the application producing the file is free to use a different layout. The performance benefit is seen whenever the read access pattern is similar to what was used on write.

The logic should also prove useful in the planned caching module.

The changes are not relevant for reading local files. In that scenario the buffer cache built into the operating system provides the same benefit for free.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: kvamme62/consolidate-algo