Skip to content
Snippets Groups Projects
Commit 75772256 authored by David Diederich's avatar David Diederich
Browse files

Merge branch 'ango2-master-legaltutorial-update-42221' into 'master' 

update compliance tutorial

See merge request !108
parents 5ecde607 b9b35643
No related branches found
No related tags found
1 merge request!108update compliance tutorial
Pipeline #36908 failed
Stage: .pre
Stage: build
Stage: containerize
Stage: scan
Stage: deploy
Stage: integration
Stage: attribution
Stage: publish
Hide whitespace changes
Inline Side-by-side
docs/tutorial/ComplianceService.md
+ 161
− 113
View file @ 75772256
## Compliance Service ## Compliance Service
## Table of Contents <a name="TOC"></a> ## Table of Contents <a name="TOC"></a>
* [Introduction](#Introduction) * [Introduction](#Introduction)
* [API usage](#API-usage) * [API usage](#API-usage)
* [What is a LegalTag](#What-is-a-LegalTag) * [What is a LegalTag?](#What-is-a-LegalTag)
* [Ingestion workflow](#Ingestion-workflow) * [Ingestion workflow](#Ingestion-workflow)
* [Creating a LegalTag](#Creating-a-LegalTag) * [Creating a LegalTag](#Creating-a-LegalTag)
* [LegalTag properties](#LegalTag-properties) * [LegalTag properties](#LegalTag-properties)
* [Creating a Record](#Creating-a-Record) * [Creating a Record](#Creating-a-Record)
* [What are Derivatives](#What-are-Derivatives) * [What are Derivatives?](#What-are-Derivatives)
* [Validating a LegalTag](#Validating-a-LegalTag) * [Validating a LegalTag](#Validating-a-LegalTag)
* [Updating a LegalTag](#Updating-a-LegalTag) * [Updating a LegalTag](#Updating-a-LegalTag)
* [Compliance on consumption](#Compliance-on-consumption) * [Compliance on consumption](#Compliance-on-consumption)
...@@ -16,6 +17,7 @@ ...@@ -16,6 +17,7 @@
* [Permissions](#Permissions) * [Permissions](#Permissions)
## Introduction<a name="Introduction"></a> ## Introduction<a name="Introduction"></a>
This document covers how to remain compliant at the different stages of the data lifecycle inside the Data Ecosystem. This document covers how to remain compliant at the different stages of the data lifecycle inside the Data Ecosystem.
1. When ingesting data 1. When ingesting data
...@@ -31,54 +33,45 @@ Records have a Legal section in their schema and this is where the compliance is ...@@ -31,54 +33,45 @@ Records have a Legal section in their schema and this is where the compliance is
Further details can be found in the [Creating a Record](#Creating-a-Record) section. Further details can be found in the [Creating a Record](#Creating-a-Record) section.
## API usage<a name="API-usage"></a> ## API usage<a name="API-usage"></a>
Details of our APIs including how to create and retrieve LegalTags can be found in our Portal documentation [here.](/solutions/dataecosystem/apis) Details of our APIs including how to create and retrieve LegalTags can be found [here.](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/blob/5ecde60781c5ce0c92579e87c4b30fdf9219a9ab/docs/api/compliance_openapi.yaml)
You currently need the role **users.datalake.viewers** to access the LegalTag API. When creating a LegalTag you need at least the **users.datalake.editors** role. You need the **users.datalake.admins** role to update legalTags.
The Data Ecosystem stores data in different data partitions depending on the access to those data partitions in the osdu system.
A user may have access to many data partitions in osdu e.g. a OSDU user may have access to both the OSDU data partition and a customers data partition. When a user logs into the osdu portal they choose which data partition they currently want to be active. ##### Permissions
When using the LegalTag APIs, you need to specify which data partition they currently have active access to and send it in the OSDU-data-partition-id header. | **_API_** | **_Minimum Permissions Required_** |
| --- | --- |
| Access LegalTag APIs| users.datalake.viewers |
| Create a LegalTag | users.datalake.editors |
| Update a LegalTag | users.datalake.editors |
OSDU-data-partition-id ##### Headers
The correct values can be obtained from CFS services. | **_Header_** | **_Description_** |
| --- | --- |
We use this value to work out which data partition to use. There is also a special data partition known as common | data-partition-id (Required) | Specify the desired accessible partition id.|
| correlation-id (Optional) | Used to track a single request throughout all the services it passes through. This can be a GUID on the header with a key. If you are the service initiating the request, you should generate the id. Otherwise, you should just forward it on in the request. |
OSDU-data-partition-id: common
This has all public data in the Data Ecosystem. Users always have access to this as well as their current active data partition.
Currently you can only specify 1 data partition Id value at a time when using the Legal APIs. If you want to retrieve all LegalTags from both the user's data partition and the common data partition, you need to do 2 separate requests, changing the header value used in each. The Data Ecosystem stores data in different data partitions, depending on the access to those data partitions in the OSDU system. A user may have access to one or more data partitions.
You can also send a correlation id as a header so that a single request can be tracked throughout all the services it passes through. This can be a GUID on the header with a key
OSDU-Correlation-Id 1e0fef08-22fd-49b1-a5cc-dffa21bc0b70
If you are the service initiating the request, you should generate the id. Otherwise, you should just forward it on in the request.
[Back to table of contents](#TOC) [Back to table of contents](#TOC)
## What is a LegalTag?<a name="What-is-a-LegalTag"></a> ## What is a LegalTag?<a name="What-is-a-LegalTag"></a>
A LegalTag is the entity that represents the legal status of data in the Data Ecosystem. It is a collection of *properties* that governs how the data can be consumed and ingested. A LegalTag is the entity that represents the legal status of data in the Data Ecosystem. It is a collection of *properties* that governs how the data can be consumed and ingested.
A legal tag is required for data ingestion. Therefore, creation of a legal tag is a necessary first step if there isn't a legal tag already exists for use with the ingested data. The LegalTag name is used for reference. A legal tag is required for data ingestion. Therefore, creation of a legal tag is a necessary first step if there isn't a legal tag already exists for use with the ingested data. The LegalTag name needs to be assigned to the LegalTag during creation, and is used for reference. The name is the unique identifier for the LegalTag that is used to access it.
When data is ingested, it is assigned the LegalTag *name*. This name is checked for a corresponding valid LegalTag in the system. A valid LegalTag means it exists and has not expired. If a LegalTag is invalid, the data is rejected. When data is ingested, it is assigned the LegalTag *name*. This name is checked for a corresponding valid LegalTag in the system. A valid LegalTag means it exists and has not expired. If a LegalTag is invalid, the data is rejected. For instance, we may not allow ingestion of data from a certain country, or we may not allow consumption of data that has an expired contract.
For instance, we may not allow ingestion of data from a certain country, or we may not allow consumption of data that has an expired contract. In the same manner, the ingested data will be invalidated (soft-deleted) when the legal tag expires, as it would no longer be compliant.
A name needs to be assigned to the LegalTag during creation. The name is a unique identifier for the LegalTag that is used to access it.
## Ingestion workflow<a name="Ingestion-workflow"></a> ## Ingestion workflow<a name="Ingestion-workflow"></a>
![API Security - High level](https://storage.googleapis.com/devportal-live-public/IngestionSequenceDiagram.png) ![API Security - High level](https://storage.googleapis.com/devportal-live-public/IngestionSequenceDiagram.png)
[](url)
The above diagram shows the typical sequence of events of a data ingestion. The important points to highlight are as follow: The above diagram shows the typical sequence of events of a data ingestion. The important points to highlight are as follow:
* It is the clients' responsibility to create a LegalTag. LegalTag validation happens at this point. * It is the ingestor's responsibility to create a LegalTag. LegalTag validation happens at this point.
* The Storage service validates the LegalTag for the data being ingested. * The Storage service validates the LegalTag for the data being ingested.
* Only after validating a LegalTag exists can we ingest data. No data should be stored at any point in the Data Ecosystem that does not have a valid LegalTag. * Only after validating a LegalTag exists can we ingest data. No data should be stored at any point in the Data Ecosystem that does not have a valid LegalTag.
...@@ -86,30 +79,16 @@ The above diagram shows the typical sequence of events of a data ingestion. The ...@@ -86,30 +79,16 @@ The above diagram shows the typical sequence of events of a data ingestion. The
Any data being ingested needs a LegalTag associated with it. You can create a LegalTag by using the POST LegalTag API e.g. Any data being ingested needs a LegalTag associated with it. You can create a LegalTag by using the POST LegalTag API e.g.
POST /api/legal/v1/legaltags POST /api/legal/v1/legaltags
{
"name": "demo-legaltag",
"description": "A legaltag used for demonstration purposes.",
"properties": {
"countryOfOrigin":["US"],
"contractId": "No Contract Related",
"expirationDate": "2099-01-01",
"dataType":"Public Domain Data",
"originator":"OSDU",
"securityClassification":"Public",
"exportClassification":"EAR99",
"personalData":"No Personal Data"
}
}
<details><summary>Curl</summary> <details><summary>Curl</summary>
``` ```
curl --request POST \ curl --request POST \
--url 'https://api.osdu.[osdu].org/de/legal/v1/legaltags' \ --url '/api/legal/v1/legaltags' \
--header 'accept: application/json' \ --header 'accept: application/json' \
--header 'authorization: Bearer <JWT>' \ --header 'authorization: Bearer <JWT>' \
--header 'content-type: application/json' \ --header 'content-type: application/json' \
--header 'OSDU-data-partition-id: common' \ --header 'data-partition-id: opendes' \
--data '{ --data '{
"name": "demo-legaltag", "name": "demo-legaltag",
"description": "A legaltag used for demonstration purposes.", "description": "A legaltag used for demonstration purposes.",
...@@ -125,22 +104,27 @@ curl --request POST \ ...@@ -125,22 +104,27 @@ curl --request POST \
} }
}' }'
``` ```
</details> </details>
It is good practice for LegalTag names to be clear and descriptive of the properties it represents, so it would be easy to discover and to associate to the correct data with it. Also, the description field is a free form optional field to allow for you to add context to the LegalTag, making easier to understand and retrieve over time. It is good practice for LegalTag names to be clear and descriptive of the properties it represents, so it would be easy to discover and to associate to the correct data with it. Also, the description field is a free form optional field to allow for you to add context to the LegalTag, making easier to understand and retrieve over time.
When creating LegalTags, the name is automatically prefixed with the data partition Id that is sent in the request. So in the example above, if the given OSDU-data-partition-id header value is **common**, then the actual name of the LegalTag would be **common-demo-legaltag**. When creating LegalTags, the name is automatically prefixed with the data-partition-name that is assigned to the partition. So in the example above, if the given data-partition-name is **mypartition**, then the actual name of the LegalTag would be **mypartition-demo-legaltag**.
Valid values: The legalTag name needs to be between 3 and 100 characters and only alphanumeric characters and hyphens are allowed
To help with LegalTag creation, it is advised to use the Get LegalTag Properties API to obtain the allowed properties before creating a legal tag. This returns the allowed values for many of the LegalTag properties. To help with LegalTag creation, it is advised to use the Get LegalTag Properties API to obtain the allowed properties before creating a legal tag. This returns the allowed values for many of the LegalTag properties.
## LegalTag properties<a name="LegalTag-properties"></a> ## LegalTag properties<a name="LegalTag-properties"></a>
Below are details of the properties you can supply when creating a LegalTag along with the values you can use. The allowed properties values can be data partition specific. Valid values associated with the property are shown. All values are mandatory unless otherwise stated. Below are details of the properties you can supply when creating a LegalTag along with the values you can use. The allowed properties values can be data partition specific. Valid values associated with the property are shown. All values are mandatory unless otherwise stated.
You can get the data partition's specific allowed properties values by using LegalTag Properties api e.g. You can get the data partition's specific allowed properties values by using LegalTag Properties API e.g.
GET /api/legal/v1/legaltags:properties GET /api/legal/v1/legaltags:properties
Example 200 Response <details><summary>Example 200 Response</summary>
```
{ {
"countriesOfOrigin": { "countriesOfOrigin": {
"TT": "Trinidad and Tobago", "TT": "Trinidad and Tobago",
...@@ -164,63 +148,77 @@ You can get the data partition's specific allowed properties values by using Leg ...@@ -164,63 +148,77 @@ You can get the data partition's specific allowed properties values by using Leg
... ...
}, },
"securityClassifications": ["Private", "Public", "Confidential"], "securityClassifications": ["Private", "Public", "Confidential"],
"exportClassificationControlNumbers": ["No License Required", "Not - Technical Data", "EAR99"], "exportClassificationControlNumbers": ["No License Required", "Not - Technical Data", "EAR99", "0A998"],
"personalDataTypes": ["Personally Identifiable", "No Personal Data"] "personalDataTypes": ["Personally Identifiable", "No Personal Data"]
} }
```
</details>
<details><summary>Curl</summary> <details><summary>Curl</summary>
``` ```
curl --request GET \ curl --request GET \
--url 'https://api.osdu.[osdu].org/de/legal/v1/legaltags:properties' \ --url '/api/legal/v1/legaltags:properties' \
--header 'accept: application/json' \ --header 'accept: application/json' \
--header 'authorization: Bearer <JWT>' \ --header 'authorization: Bearer <JWT>' \
--header 'content-type: application/json' \ --header 'content-type: application/json' \
--header 'OSDU-data-partition-id: common' \ --header 'data-partition-id: opendes' \
``` ```
</details> </details>
#### Country of origin #### Country of origin
Valid values: An array of ISO Alpha-2 country code. This is normally one value but can be more. This is required. Valid values: An array of ISO Alpha-2 country code. This is normally one value but can be more. This is required.
Notes: This is the country from where the data originally came, NOT from where the data was sent. The list of allowed countries is below. If ingesting Third Party Data, you can ingest data from any country that is not embargoed, if you have a valid contract associated with it that allows for this. Notes: This is the country from where the data originally came, NOT from where the data was sent. The list of allowed countries is below. If ingesting Third Party Data, you can ingest data from any country that is not embargoed, if you have a valid contract associated with it that allows for this. This property is case sensitive.
#### Contract Id #### Contract Id
Valid values: This should be the Contract Id associated with the data or 'Unknown' or 'No Contract Related'. Valid values: This should be the Contract Id associated with the data or 'Unknown' or 'No Contract Related'.
The contract ID must be between 3 and 40 characters and only include alphanumeric values and hyphens.
Notes: This is always required for any data types. Notes: This is always required for any data types. This property is case sensitive.
#### Expiration date #### Expiration date
Valid values: Any date in the future in the format yyyy-MM-dd (e.g. 2099-12-25) or empty. Valid values: Any date in the future in the format yyyy-MM-dd (e.g. 2099-12-25) or empty.
Notes: This sets the inclusive date when the LegalTag expires and the data it relates to is no longer usable in the Data Ecosystem. This normally is taken from the physical contracts expiration date i.e. when you supply a contract ID. This is non mandatory field, but is required for certain types of data e.g. 3rd party. If the field is not set it will be autopopulated with the value 9999-12-31. When the provided contract ID is "Unknown" or "No Contract Related", then the expiration date could be empty. However, if the date is provided, then it will be honored in validating the legal tag and the associated data, even when there's no contract is provided. As such, when the legal tag expires, the associated data will be soft-deleted from the Data Ecosystem.
Notes: This sets the inclusive date when the LegalTag expires and the data it relates to is no longer usable in the Data Ecosystem. This normally is taken from the physical contracts expiration date i.e. when you supply a contract ID. This is non mandatory field, but is required for certain types of data e.g. 3rd party. If the field is not set it will be autopopulated with the value 9999-12-31. This property is case sensitive.
#### Originator #### Originator
Valid values: Should be the name of the client, supplier or Self Valid values: Should be the name of the client or supplier.
Notes: This is always required. Notes: This is always required. This property is case sensitive.
#### Data type #### Data type
Valid values: 'OSDU Data', 'Public Domain Data', 'EHC Data', 'Index Data', 'Third Party Data', 'Client Data''.
| dataType | Data Residency Restriction |
| ----------- | -------- |
| "Public Domain Data" | "public data, no contract required" |
| "First Party Data" | "partition owner's data, no contract required" |
| "Second Party Data" | "client data, contract is required" |
| "Third Party Data" | "contract required" |
| "Transferred Data" | "EHC/Index data, no contract required" |
Notes: Different data types are allowed dependent on the data partitions e.g. vendor partitions have different governing rules as opposed to standard partitions. To list the allowed data types for your data partition use the [LegalTag Properties](#LegalTag-properties). 'Third Party Data' is allowed ONLY with a contract ID and expiration date set. 'Client Data' is the ONLY allowed value in Client data partitions, and 'Client Data' can be allowed in OSDU data partitions under exemption, in which case the contract ID and expiration date are required; contract ID and expiration date are not required if ingesting 'Client Data" in client data partitions. Notes: Different data types are allowed dependent on the data partitions e.g. vendor partitions have different governing rules as opposed to standard partitions. To list the allowed data types for your data partition use the [LegalTag Properties](#LegalTag-properties).
#### Security classification #### Security classification
Valid values: 'Public', 'Private', 'Confidential' Valid values: 'Public', 'Private', 'Confidential'
Notes: This is the OSDU standard security classification for the data. We currently do not allow 'Secret' data to be stored in the Data Ecosystem. Notes: This is the standard security classification for the data. We currently do not allow 'Secret' data to be stored in the Data Ecosystem. This property is NOT case sensitive.
#### Export classification #### Export classification
Valid values: 'EAR99', 'Not - Technical Data', 'No License Required' Valid values: '0A998'(0 as Zero), 'EAR99', 'Not - Technical Data', 'No License Required'
Notes: We currently only allow data with the ECCN classification 'EAR99' Notes: We currently only allow data with the ECCN classification 'EAR99' and '0A998'(0 as Zero). This property is NOT case sensitive.
#### Personal data #### Personal data
Valid values: 'Personally Identifiable', 'No Personal Data' Valid values: 'Personally Identifiable', 'No Personal Data'
Notes: We do not currently allow data that is 'Sensitive Personal Information' and this should not be ingested. Notes: We do not currently allow data that is 'Sensitive Personal Information' and this should not be ingested. This property is NOT case sensitive.
[Back to table of contents](#TOC) [Back to table of contents](#TOC)
...@@ -236,40 +234,48 @@ When creating a Record, the following needs to be assigned for legal compliance: ...@@ -236,40 +234,48 @@ When creating a Record, the following needs to be assigned for legal compliance:
Below is a full example of the payload needed when creating a Record. The *legal* section shows what is required. Below is a full example of the payload needed when creating a Record. The *legal* section shows what is required.
<details><summary>Details</summary>
```
[{ [{
"acl": { "acl": {
"owners": [ "owners": [
"data.default.owners@common.osdu.[osdu].org" "data.default.owners@{datapartition}.{domain}.org"
], ],
"viewers": [ "viewers": [
"data.default.viewers@common.osdu.[osdu].org" "data.default.viewers@{datapartition}.{domain}.org"
] ]
}, },
"data": { "data": {
"count": 123456789 "count": 123456789
}, },
"id": "common:id:123456789", "id": "opendes:id:123456789",
"kind": "common:welldb:wellbore:1.0.0", "kind": "opendes:welldb:wellbore:1.0.0",
"legal" :{ "legal" :{
"legaltags": [ "legaltags": [
"common-demo-legaltag" "opendes-demo-legaltag"
], ],
"otherRelevantDataCountries": ["US"] //the physical location of the person ingesting the data "otherRelevantDataCountries": ["US"] //the physical location of the person ingesting the data
} }
}] }]
```
</details>
* legaltags - This section represents the names of the LegalTag(s) associated with the Record. This has to be supplied when the Record represents raw or source data (i,e, not derivative data) * legaltags - This section represents the names of the LegalTag(s) associated with the Record. This has to be supplied when the Record represents raw or source data (i,e, not derivative data)
* otherRelevantDataCountries - This is the Alpha-2 country codes for the country the data was ingested from and the country where the data is located in Data Ecosystem. * otherRelevantDataCountries - This is the Alpha-2 country codes for the country the data was ingested from and the country where the data is located in Data Ecosystem. otherRelevantDataCountries is not part of the LegalTag(s). It is part of the legal property of the record. The location of the data center, in which the record is stored is automatically added to the otherRelevantDataCountries list when the record is created. This location depends on the environment/region that the partition locates.
You can get the list of all valid LegalTags using the Get LegalTags API method. You can use this to help assign only valid LegalTags to data when ingesting. You can get the list of all valid LegalTags using the Get LegalTags API method. You can use this to help assign only valid LegalTags to data when ingesting.
GET /api/legal/v1/legaltags?valid=true GET /api/legal/v1/legaltags?valid=true
Example 200 Response <details><summary>Example 200 Response</summary>
```
{ {
"legalTags": [ "legalTags": [
{ {
"name": "OSDU-ehc-public", "name": "osdu-thirdparty-public",
"description": "", "description": "",
"properties": { "properties": {
"countryOfOrigin": [ "countryOfOrigin": [
...@@ -278,14 +284,14 @@ You can get the list of all valid LegalTags using the Get LegalTags API method. ...@@ -278,14 +284,14 @@ You can get the list of all valid LegalTags using the Get LegalTags API method.
"contractId": "A1234", "contractId": "A1234",
"expirationDate": "2099-01-25", "expirationDate": "2099-01-25",
"originator": "OSDU", "originator": "OSDU",
"dataType": "EHC Data", "dataType": "Third Party Data",
"securityClassification": "Public", "securityClassification": "Public",
"personalData": "No Personal Data", "personalData": "No Personal Data",
"exportClassification": "EAR99" "exportClassification": "EAR99"
} }
}, },
{ {
"name": "OSDU-welldb-public", "name": "osdu-welldb-public",
"description": "", "description": "",
"properties": { "properties": {
"countryOfOrigin": [ "countryOfOrigin": [
...@@ -294,7 +300,7 @@ You can get the list of all valid LegalTags using the Get LegalTags API method. ...@@ -294,7 +300,7 @@ You can get the list of all valid LegalTags using the Get LegalTags API method.
"contractId": "AB123", "contractId": "AB123",
"expirationDate": "2099-12-25", "expirationDate": "2099-12-25",
"originator": "OSDU", "originator": "OSDU",
"dataType": "OSDU Data", "dataType": "Public Domain Data",
"securityClassification": "Public", "securityClassification": "Public",
"personalData": "No Personal Data", "personalData": "No Personal Data",
"exportClassification": "EAR99" "exportClassification": "EAR99"
...@@ -304,17 +310,21 @@ You can get the list of all valid LegalTags using the Get LegalTags API method. ...@@ -304,17 +310,21 @@ You can get the list of all valid LegalTags using the Get LegalTags API method.
... ...
... ...
} }
```
</details>
<details><summary>Curl</summary> <details><summary>Curl</summary>
``` ```
curl --request GET \ curl --request GET \
--url 'https://api.osdu.[osdu].org/de/legal/v1/legaltags?valid=true' \ --url '/api/legal/v1/legaltags?valid=true' \
--header 'accept: application/json' \ --header 'accept: application/json' \
--header 'authorization: Bearer <JWT>' \ --header 'authorization: Bearer <JWT>' \
--header 'content-type: application/json' \ --header 'content-type: application/json' \
--header 'OSDU-data-partition-id: common' \ --header 'data-partition-id: opendes' \
``` ```
</details> </details>
## What are Derivatives?<a name="What-are-Dervatives"></a> ## What are Derivatives?<a name="What-are-Dervatives"></a>
...@@ -334,27 +344,32 @@ When creating Records that represent derivative data, the following must be assi ...@@ -334,27 +344,32 @@ When creating Records that represent derivative data, the following must be assi
Below is an example of the minimum number of fields required to ingest a derivative Record. Below is an example of the minimum number of fields required to ingest a derivative Record.
<details><summary>Details</summary>
```
[{ [{
"acl": { "acl": {
"owners": [ "owners": [
"data.default.owners@common.osdu.[osdu].org" "data.default.owners@{datapartition}.{domain}.org"
], ],
"viewers": [ "viewers": [
"data.default.viewers@common.osdu.[osdu].org" "data.default.viewers@{datapartition}.{domain}.org"
] ]
}, },
"data": { "data": {
"count": 123456789 "count": 123456789
}, },
"id": "common:id:123456789", "id": "opendes:id:123456789",
"kind": "common:welldb:wellbore:1.0.0", "kind": "opendes:welldb:wellbore:1.0.0",
"legal" :{ "legal" :{
"otherRelevantDataCountries": ["US"] //the physical location of where the derivative was created "otherRelevantDataCountries": ["US"] //the physical location of where the derivative was created
}, },
"ancestry" :{ "ancestry" :{
"parents": ["common:id:1:version", "common:id:2:version"] //the record ids and versions of the Records this derivative was created from "parents": ["opendes:id:1:version", "opendes:id:2:version"] //the record ids and versions of the Records this derivative was created from
} }
}] }]
```
</details>
As shown the parent Records are provided as well as the ORDC of where the derivative was created. The Record service takes responsibility for populating the full LegalTag and ORDC values based on the parents. As shown the parent Records are provided as well as the ORDC of where the derivative was created. The Record service takes responsibility for populating the full LegalTag and ORDC values based on the parents.
...@@ -369,40 +384,47 @@ However, there may be times you want to validate LegalTags directly. ...@@ -369,40 +384,47 @@ However, there may be times you want to validate LegalTags directly.
You can validate a LegalTag by using the LegalTag validate API supplying the names of the LegalTags you wish to validate e.g. You can validate a LegalTag by using the LegalTag validate API supplying the names of the LegalTags you wish to validate e.g.
POST /api/legal/v1/legaltags:validate POST /api/legal/v1/legaltags:validate
Body
{
"names": ["common-demo-legaltag"]
}
<details><summary>Curl</summary>
```
curl --request POST \
--url '/api/legal/v1/legaltags:validate' \
--header 'accept: application/json' \
--header 'authorization: Bearer <JWT>' \
--header 'content-type: application/json' \
--header 'data-partition-id: opendes' \
--data '{
"names": ["opendes-demo-legaltag"]
}'
```
</details>
If the LegalTag is valid, the response then looks something like this If the LegalTag is valid, the response then looks something like this
<details><summary>Details</summary>
```
{ {
"invalidLegalTags": [] "invalidLegalTags": []
} }
```
</details>
If the LegalTag is invalid, the response then looks something like this If the LegalTag is invalid, the response then looks something like this
<details><summary>Details</summary>
```
{ {
"invalidLegalTags": [ "invalidLegalTags": [
{"name":"common-demo-legaltag", "reason": "Contract expired"} {"name":"opendes-demo-legaltag", "reason": "Contract expired"}
] ]
} }
<details><summary>Curl</summary>
```
curl --request POST \
--url 'https://api.osdu.[osdu].org/de/legal/v1/legaltags:validate' \
--header 'accept: application/json' \
--header 'authorization: Bearer <JWT>' \
--header 'content-type: application/json' \
--header 'OSDU-data-partition-id: common' \
--data '{
"names": ["common-demo-legaltag"]
}'
``` ```
</details> </details>
So if you just want to check that the given LegalTag(s) are currently valid, you only have to check if the returned 'invalidLegalTags' collection is empty. So if you just want to check that the given LegalTag(s) are currently valid, you only have to check if the returned 'invalidLegalTags' collection is empty.
...@@ -415,13 +437,32 @@ One of the main cases where a LegalTag can become invalid is if a contract expir ...@@ -415,13 +437,32 @@ One of the main cases where a LegalTag can become invalid is if a contract expir
In these situations we can update LegalTags to make them valid again and so make the associated data accessible. Currently we only allow the update of the *description*, *contract ID* and *expiration date* properties. In these situations we can update LegalTags to make them valid again and so make the associated data accessible. Currently we only allow the update of the *description*, *contract ID* and *expiration date* properties.
PUT /api/legal/v1/legaltags PUT /api/legal/v1/legaltags
Body
{ <details><summary>Curl</summary>
"name": "common-demo-legaltag", //the name of the legaltag you are updating
"contractId", "AE12345" ```
"expirationDate", "2099-12-21" curl --request PUT \
--url '/api/legal/v1/legaltags' \
--header 'accept: application/json' \
--header 'authorization: Bearer <JWT>' \
--header 'content-type: application/json' \
--header 'data-partition-id: opendes' \
--data '{
"name": "opendes-demo-legaltag",
"contractId": "AE12345"
"expirationDate": "2099-12-21"
} }
}'
```
</details>
*Note: There is a difference between "querying for valid or invalid LegalTag (List Legal Tag API )" and "checking if the legalTag is Valid (Validate Legal Tag API)".
The "List Legal tag" will check whether the tag is valid right now and "Validate Legal Tag" will check whether the tag would be valid after the next once-a-day update process.
Therefore, updating the LegalTag Expiration Date will not make the record visible as valid in the Valid Tag List until its status has been updated.
The Update process validates the LegalTags and then updates the status of the LegalTag from Invalid to Valid or vice versa. This update process runs only once in a day.
[Back to table of contents](#TOC) [Back to table of contents](#TOC)
## Compliance on consumption<a name="Compliance-on-Consumption"></a> ## Compliance on consumption<a name="Compliance-on-Consumption"></a>
...@@ -429,6 +470,7 @@ As previously stated, the Records in the Storage service largely governs data co ...@@ -429,6 +470,7 @@ As previously stated, the Records in the Storage service largely governs data co
However, there are use cases where you may not use these services all the time e.g. if you have your own operational data store. In these cases you will need to check the LegalTags associated with your data are still valid before allowing consumption. For this, we have a PubSub topic that can be subscribed to. However, there are use cases where you may not use these services all the time e.g. if you have your own operational data store. In these cases you will need to check the LegalTags associated with your data are still valid before allowing consumption. For this, we have a PubSub topic that can be subscribed to.
*NOTE This topic can only be subscribed to if you deploy your service within the Data Ecosystem Google Project.
This topic has the form This topic has the form
...@@ -443,6 +485,9 @@ For more information on subscribing to PubSub topics, please use the Google docu ...@@ -443,6 +485,9 @@ For more information on subscribing to PubSub topics, please use the Google docu
## The LegalTag Changed notification<a name="The-LegalTag-Changed-notification"></a> ## The LegalTag Changed notification<a name="The-LegalTag-Changed-notification"></a>
After subscribing to the topic, you will receive notifications daily. These notifications will list all LegalTags that have changed, and whether the LegalTag has become compliant or non-compliant. After subscribing to the topic, you will receive notifications daily. These notifications will list all LegalTags that have changed, and whether the LegalTag has become compliant or non-compliant.
<details><summary>Details</summary>
```
{ {
"statusChangedTags": [ { "statusChangedTags": [ {
"changedTagName": "legaltag-name1", "changedTagName": "legaltag-name1",
...@@ -453,6 +498,9 @@ After subscribing to the topic, you will receive notifications daily. These noti ...@@ -453,6 +498,9 @@ After subscribing to the topic, you will receive notifications daily. These noti
"changedTagStatus": "incompliant" "changedTagStatus": "incompliant"
} ] } ]
} }
```
</details>
The above shows an example message sent to subscribers. It shows you receive an array of items. Each item has the LegalTag name that has changed and whether it has changed to be compliant or incompliant. The above shows an example message sent to subscribers. It shows you receive an array of items. Each item has the LegalTag name that has changed and whether it has changed to be compliant or incompliant.
...@@ -460,4 +508,4 @@ If it has become incompliant, you must make sure associated data is no longer al ...@@ -460,4 +508,4 @@ If it has become incompliant, you must make sure associated data is no longer al
If it is marked compliant, data that was not allowed for consumption can now be consumed through your services. If it is marked compliant, data that was not allowed for consumption can now be consumed through your services.
[Back to table of contents](#TOC) [Back to table of contents](#TOC)
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment