Skip to content

add validation and ignore out of order message when kind is updated

Neelesh Thakur requested to merge ignore-outdated-messages into master

Users have reported issue with data duplication with following scenario:

  1. Trigger a re-index job for kinds: both raw and WKS kind at the same time
  2. The triggering of the reindexing on raw kind also triggered WKS service to recreate the WKS instances derived from them
  3. The target schema mapping for these raw records had been updated (e.g. version 1.0 to 1.1), meaning the new versions of these records created by WKS changed the schema.
  4. Index therefore received 2 events for the same WKS record instances at similar times, one to re-index the original and another to create the new one
  5. If the one to create the new instance for WKS was processed first that would lead to duplicates appearing.

This changes-set adds validation to avoid duplication by ignoring the event when kind is updated in such cases.

Merge request reports