Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • I Ingestion Workflow
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 45
    • Issues 45
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 10
    • Merge requests 10
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe Software
  • Platform
  • Data Flow
  • Data Ingestion
  • Ingestion Workflow
  • Merge requests
  • !237

[Azure] Redis cache fixes

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Aalekh Jain requested to merge Ironman-RedisCacheFix into master Feb 07, 2022
  • Overview 2
  • Commits 11
  • Pipelines 15
  • Changes 10

Description of the original error encountered

The RedisCache class is hard coded with CompressionCodec. This codec is used for encoding and decoding purposes of key and values in redis cache.

The bug arises when the following sequence of steps takes place.

  1. Using atomic increment operation of redis cache will increment the value corresponding to the given key. Since redis cache will not have any data type, it will just increment the value assuming the value to be plain bytes (which it is not since we're using the compression codec)
  2. Using the get operation on the same key will result in EOFException as the decoding happens on the incremented value from the above step which assumed integer type/representation in the earlier step (which is the not the case since CompressionCodec will change the values from plain bytes to compressed bytes).

Gist of this problem: The problem arises from the issue that Redis has no data types and all values are just bytes while certain commands imply a particular type/representation.

Resolution

CompressionCodec uses JSONCodec, which will not change the plain bytes representation of the data stored in redis cache. Hence upon using JSONCodec, we get rid of the EOFException error arising when using get operation after increment operation.

However, there is no way we can provide custom codec to the existing redis cache method. Hence this MR in os core common adds ability to provide (or override) the codec that will be used for serialization and deserialization purposes in RedisCache. Link to the MR: osdu/platform/system/lib/core/os-core-common!148 (merged). Once the MR is merged in os core common, one can keep using the default codec (CompressionCodec) OR can override the codec as well as per the requirements if the need be.

cc: @kibattul @krveduru

Edited Mar 03, 2022 by Aalekh Jain
Assignee
Assign to
Reviewer
Request review from
Time tracking
Source branch: Ironman-RedisCacheFix