ADR: OSDU Legal Tags - send notification before tag expire
Introduction
The OSDU Legal service is designed to manage legal and compliance aspects related to data in the OSDU data platform, primarily in the context of the oil and gas industry. It plays a crucial role in ensuring that data stored and exchanged within the OSDU platform complies with legal requirements and industry standards.
OSDU Legal service operates on LegalTag entities. A LegalTag is the entity that represents the legal status of data in the Data Ecosystem. It is a collection of properties that governs how the data can be consumed and ingested. A legal tag is required for data ingestion. Therefore, creation of a legal tag is a necessary first step if there isn't a legal tag that already exists for use with the ingested data. User can provide expiration date for LegalTag. When expiration date is provided, then the ingested data will be invalidated (soft-deleted) when the legal tag expires, as it would no longer be compliant.
From LegalTag documentation
In the same manner, the ingested data will be invalidated (soft-deleted) when the legal tag expires, as it would no longer be compliant.
This creates use-case where it is required to regularly maintain them, check expiration date etc. Currently OSDU Legal service can provide notification using OSDU Notification service, this notification is being triggered on LegalTag expiration. As I mentioned before this renders all data assigned to this LegalTag as soft-deleted (inaccessible) and may introduce delays in processing data.
To avoid this situation it is required to add a new way of notifying users about possible future expiration of a LegalTag. This functionality should be configurable on what time periods we are interested in. Also multiple notifications should be sent.
Status
-
Proposed -
Trialing -
Under review -
Approved -
Retired
Context & Scope
(1) OSDU Legal Service
Currently, the only method to monitor whether a LegalTag has expired is to subscribe to a topic in OSDU Notification service for LegalTag Change. This will send a daily list of all LegalTags that have changed since the previous notification. This list includes all changes to attributes of LegalTag, also whether the LegalTag has become complaint or non-compliant.
Sample notification:
{ "statusChangedTags": [ { "changedTagName": "legaltag-name1", "changedTagStatus": "compliant" }, { "changedTagName": "legaltag-name2", "changedTagStatus": "incompliant" } ] }
For more information on The LegalTag Changed notification please follow OSDU Forum documentation
Currently OSDU Legal services runs daily scheduled task to generate list of all changes to LegalTags. This provides us good indication that that data are checked regularly. We can use this mechanism to see what LegalTags are close to expiration and emit separate notification to users.
Proposed notification topic legaltagclosetoexpiretopic
Proposed notification payload for LegalTags close to expiring:
{ "closeToExpireTags": [ { "tagName": "legaltag-name1", "expirationDate": "2026-11-25" }, { "tagName": "legaltag-name2", "expirationDate": "2026-01-11" }, ] }
Time to expire should be configurable on deployment level. We should be able to provide list of all time periods before LegalTag expires. This will be achieved by introducing new environment variable LEGALTAG_EXPIRATION. Format is comma separated values which consists of number and letter suffix, for example: 3d,2w,1m
Where suffix letter should represent this time periods:
-
#d- # number of days -
#w- # number of weeks -
#m- # number of months
Most of changes should be limited only to legal service, current job is present here: https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/blob/master/legal-core/src/main/java/org/opengroup/osdu/legal/jobs/LegalTagStatusJob.java
Another part that need to be updated is Notification Publisher, core service consist only of interface, and each Cloud Provider need to implement their connector
Possible changes:
- Update current cron job with new functionality of close to expire tags
- Update filters to include date range to filters ListLegalTagArgs
- Provide way to configure multiple time periods that would be used for searching Legal Tags close to expire
- Create Notification publisher to provide list of Legal Tags close to expire, this should take into account previous change
(2) OSDU Notification Service
No changes required, but OSDU Notification Service needs to be able to subscribe to new topic, this will require registering new topic and also OSDU Notification needs to be implemented in all environments to receive this notification.
(3) OSDU Register
Need to register new topic for close to expire notifications: legaltagclosetoexpiretopic
Decision
To be decided