Re-implement Try/Block per-record to prevent wasted batches on single-record-error
Type of change
-
Bug Fix -
Feature
Please provide link to gitlab issue or ADR(Architecture Decision Record)
Does this introduce a change in the core logic?
- [YES/NO]
Does this introduce a change in the cloud provider implementation, if so which cloud?
-
AWS -
Azure -
GCP -
IBM
Does this introduce a breaking change?
- [YES/NO]
What is the current behavior?
- If a single record produces error during ingestion, the entire batch is skipped
What is the new/expected behavior?
- If a single record produces error during ingestion, only the record is skipped
Have you added/updated Unit Tests and Integration Tests?
N/A