Unable to use custom codec with RedisCache
Currently, the codec that is used for the RedisCache
is hard coded to CompressionCodec
. However, there are use cases where one might want to use custom codecs (primarily used for desired serialization and de-serialization of the redis keys or values). Hence, a separate method which can be overridden needs to be introduced using which one can use the cusom codecs for RedisCache
if the need be.
One such use case can be found here in ingestion workflow: https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/blob/441b6489dc954bee1df890b45d434651c219810a/provider/workflow-azure/src/main/java/org/opengroup/osdu/workflow/provider/azure/cache/ActiveDagRunsRedisCache.java#L30 (The method here uses JSONCodec
instead of CompressionCodec
)
This is the corresponding MR for the above use case which also contains the detailed description of why this change is required: osdu/platform/data-flow/ingestion/ingestion-workflow!237 (merged)
Described below is the scenario where using CompressionCodec
will result into erroneous scenario and one might want to use another codec such as JSONCodec
- Assuming the datatype of key in RedisCache is
String
and the datatype of value in RedisCache isInteger
- Using atomic increment operation of redis cache will increment the value corresponding to the given key. Since redis cache internally does not have any data type, it will just increment the value assuming the value to be plain bytes (which it is not since we're using the compression codec)
- Using the get operation on the same key will result in EOFException as the decoding happens on the incremented value from the above step which assumed integer type/representation in the earlier step (which is the not the case since CompressionCodec will change the values from plain bytes to compressed bytes).
Using JSONCodec
can resolve this issue. This is how it's done here: https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/blob/441b6489dc954bee1df890b45d434651c219810a/provider/workflow-azure/src/main/java/org/opengroup/osdu/workflow/provider/azure/cache/ActiveDagRunsRedisCache.java#L30
The changes are targetted in this MR: !148 (merged)