Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Register
  • Sign in
  • S Segy to Zgy Conversion
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
    • Locked files
  • Issues 3
    • Issues 3
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe SoftwareOpen Subsurface Data Universe Software
  • Platform
  • Data Flow
  • Data IngestionData Ingestion
  • Segy to Zgy Conversion
  • Merge requests
  • !171

Push-Pull pattern implementation for conversion and data loading (performance improvement)

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Borys Oliynyk requested to merge slb-master-push-pull-multibulk-reading into slb-master Nov 22, 2022
  • Overview 7
  • Commits 18
  • Pipelines 21
  • Changes 62

Type of change

  • Bug Fix
  • Feature

Does this introduce a change in the core logic?

  • [No]

Does this introduce a change in the cloud provider implementation, if so which cloud?

  • AWS
  • Azure
  • GCP
  • IBM

Updates description

The following changes have been done:

  • updated unit tests to match the Push Pull pattern implementation

  • implemented Push Pull pattern when next data are being loaded while the previous are being converted

  • added read chunk support to buffered readers class family

  • 'segysdk::bulkaccess::SegyReaderFactory' sets the chunk size for bulk readers

  • renamed to 'segysdk::segyindex::SegyIndex::getConstantTraceWeightingFactor'

  • modified interface 'segysdk::segyindex::SegyIndex::std::unique_ptrbulkaccess::SegyBulkReader createMultiBulkReader(std::shared_ptrsegyindex::SegyIndex segyIndex, size_t numberOfTraces, size_t bulksCount)'

  • added several members to interface 'segysdk::bulkaccess::SegyBulkReader'

    added:

    • size_t getBufferSize() const
    • void setBufferLoadChunk(size_t newChunk)
    • void LoadDataToCache(size_t fileOffset, size_t dataSize)

    modified (return value):

    • std::optional<size_t> setReadOffsetHint(size_t inlineIndex, size_t xlineIndex, size_t sampleIndex) added 'SD_CHUNK_SIZE' env variable
  • added 'segysdk::CacheSizeOption::chunkSize()' member

  • added interface 'segysdk::bulkaccess::BulkIndex::TraceIndexer::findTraceByIndex(.)'

Edited Nov 22, 2022 by Borys Oliynyk
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: slb-master-push-pull-multibulk-reading