ADR: Additional attributes in Storage record delete notification events

Additional attributes in Storage record delete notification events

Status

• Proposed
• Trialing
• Under review
• Approved
• Retired

Context & Scope

The storage service provides 2 ways to delete a record. One way is to logically delete the record in which the record with same id can be revived later because its version history is maintained and the other one is to purge the record in which case, the record's version history is deleted too. In both types of deletions, the record cannot be accessed using storage or search service.

The storage service provides separate APIs for logical deletion (POST /records/{id}:delete)and purging of records (DELETE /records/{id}).

This difference in behavior of deleting the records is not clear from the information received from notification service on record delete. In both the cases, it has 3 fields like below :

{
  "id":"opendes:Wellbore:id1",
  "kind":"opendes:wks:master-data--Wellbore:1.0.0",
  "op":"delete"
}

If the storage service can publish details about whether the record is being deleted logically or completely purged, it will be useful for applications who are consumers of these notification events. For example, if a record is being logically deleted then the derived records can also be logically deleted and same for the purge scenario. However these details are not yet published in the current notification events.

Tradeoff Analysis

To know whether the record is completely purged or logically deleted, consumer applications would have to make a call to storage API GET /records/versions/{id}/. This API returns the previous versions of the record if the record is only logically deleted and returns a 404 if the record is hard deleted. Calling this API for every deleted record by a consumer application will require additional call to storage, which will increase load on storage service. Instead if the storage service can provide this information as a part of record change events, this processing at consumers side can be saved.

Consumer applications can choose to always purge the derived records/data and recreate the records/data when parent record is revived. However consumer application will lose the version history of the derived record once it is purged.

Consumer applications can choose to always logically delete the derived records and recreate the record when parent record is revived. However in this case there will be a mismatch between number of versions available for original record and the derived record.

Decision

We can enhance current behavior of storage service and provide more information in the record delete event. Downstream services must be updated to consume the updated events structure. Currently the record delete event is structured as below

{
  "id":"opendes:Wellbore:id1",
  "kind":"opendes:wks:master-data--Wellbore:1.0.0",
  "op":"delete"
}

We would like to publish below additional information when the record is deleted i.e. when op is delete.

New attribute deletionType: to show if the record is deleted logically:

{
    "id":"opendes:Wellbore:id1",
    "kind":"opendes:wks:master-data--Wellbore:1.0.0",
    "op":"delete",
    "deletionType":"soft"
}

New attribute deletionType: to show if the record is purged:

{
    "id":"opendes:Wellbore:id1",
    "kind":"opendes:wks:master-data--Wellbore:1.0.0",
    "op":"delete",
    "deletionType":"hard"
}

The consumer services of this event can accordingly process the records based on deletionType. For example if the record is soft-deleted, then the consumer service will soft-delete the derived records and same in case of hard-deletion of the record.

Consequences

• Record delete event needs to send new attribute suggested above in case of records deletion.
• Register service and accompanying documentation needs to update the example record delete event information to reflect the new attributes in list topics API.
• Consumer applications need to react to the updated event information from OSDU notification service.