Skip to content

Fix retry for deleted records when index not exist

Rustam Lotsmanenko (EPAM) requested to merge add-not-retrybile-tasks into master

Description:

When the indexer receives an event with Op type DELETE, it deletes provided records at elasticsearch. During integration tests scenario when index completely deleted at elasticsearch are often, and in that case, if index receives DELETE events for specific records in that index, they will always fail:

{
    "data": "[{\"id\":\"osdu:work-product-component--WellboreMarkerSet:60182840a28944bb81d81cc4d4c60136\",\"kind\":\"testtest:wks:work-product-component--WellboreMarkerSet:1.0.0\",\"op\":\"delete\"}]",
    "attributes": {
        "account-id": "{{data-partition-id}}"
    }
}

But due to NOT_FOUND response from elasticsearch this task will go for the next round of retry

"{auditLog={resources=[RecordStatus(id=osdu:inttest:164613302639927, kind=osdu:ds:inttest:1.0.1646133014242, operationType=delete, status=FAIL, message=[osdu-ds-inttest-1.0.1646133014242] ElasticsearchException[Elasticsearch exception [type=index_not_found_exception, reason=no such index [osdu-ds-inttest-1.0.1646133014242]]])], action=DELETE, actionId=IN003, message=Failed deleting record in index, user=workload-indexer-gcp@nice-etching-277309.iam.gserviceaccount.com, status=FAILURE}} {account-id=osdu, correlation-id=8729b1d4-17c9-4141-9257-5fd63ec9c373, data-partition-id=osdu}"

Currently, this forms a cycle at GCP env with endlessly task retry.

How to test:

Does functionality was tested and how?

Changes include:

  • Refactor (a non-breaking change that improves code maintainability).
  • Bugfix (a non-breaking change that solves an issue).
  • New feature (a non-breaking change that adds functionality).
  • Breaking change (a change that is not backward-compatible and/or changes current functionality).

Changes in:

  • Common code

Dev Checklist:

  • Added Unit Tests, wherever applicable.
  • Updated the Readme, if applicable.
  • Existing Tests pass
  • Verified functionality locally
  • Self Reviewed my code for formatting and complex business logic.
Edited by Rustam Lotsmanenko (EPAM)

Merge request reports