Skip to content
Snippets Groups Projects
Commit 1c7346e3 authored by Alok Joshi's avatar Alok Joshi
Browse files

update tutorial documentation

parent 3ad95e7d
No related branches found
No related tags found
2 merge requests!744Upgraded packages to mitigated vulns in netty, guava, snakeyaml,!639Extend patch API to support data block modification
Pipeline #171692 failed
Pipeline: Storage

#171693

    ......@@ -34,10 +34,12 @@
    - [Parameters <a name="parameters"></a>](#parameters-2)
    - [Delete record <a name="Delete-record"></a>](#delete-record)
    - [Delete records <a name="Delete-records"></a>](#delete-records)
    - [Patch api <a name="patch-api"></a>](#patch-api)
    - [Replace Tags, ACLs and Legal Tags <a name="patch-api-metadata-bulk-replace"></a>](#patch-api-metadata-bulk-replace)
    - [Add Tags, ACLs and Legal Tags <a name="patch-api-metadata-bulk-add"></a>](#patch-api-metadata-bulk-add)
    - [Remove Tags, ACLs and Legal Tags <a name="patch-api-metadata-bulk-remove"></a>](#patch-api-metadata-bulk-remove)
    - [Metadata update api <a name="metadata-update-api"></a>](#metadata-update-api)
    - [Replace Tags, ACLs and Legal Tags <a name="metadata-update-replace"></a>](#metadata-update-replace)
    - [Add Tags, ACLs and Legal Tags <a name="metadata-update-add"></a>](#metadata-update-add)
    - [Remove Tags, ACLs and Legal Tags <a name="metadata-update-remove"></a>](#metadata-update-remove)
    - [Records patch api <a name="records-patch-api"></a>](#records-patch-api)
    - [Differences compared to metadata update api <a name="patch-apis-diff"></a>](#patch-api-diff)
    - [Using service accounts to access Storage APIs <a name="Service-accounts"></a>](#using-service-accounts-to-access-storage-apis)
    - [Using skipdupes <a name="skipdupes"></a>](#using-skipdupes)
    - [Support for GeoJSON types <a name="geojson-support"></a>](#support-for-geojson-types)
    ......@@ -426,9 +428,9 @@ curl --request POST \
    ```
    </details>
    ## Patch api <a name="patch-api"></a>
    ## Metadata update api <a name="metadata-update-api"></a>
    Bulk Update API allows the update of records metadata in batch. It takes an array of record ids with/without version
    This API allows update of records metadata in batch. It takes an array of record ids with/without version
    numbers with a maximum number of 500, and updates properties specified in the operation path with value and operation type
    provided. If a version number is provided, updates will be applied to the specific version of the record. If not, the
    latest version of the record will be updated. Users need to specify the corresponding data partition id in the header as
    ......@@ -452,7 +454,7 @@ Bulk Update API has the following response codes:
    PATCH /api/storage/v2/records
    ```
    ### Replace Tags, ACLs and Legal Tags <a name="patch-api-metadata-bulk-replace"></a>
    ### Replace Tags, ACLs and Legal Tags <a name="metadata-update-replace"></a>
    In the "replace" operation, property value in "path" would be fully replaced by values provided in the "value" field. If
    we need to replace tags ops.value should be colon separated string value.
    ......@@ -512,7 +514,7 @@ curl --request PATCH \
    </details>
    ### Add Tags, ACLs and Legal Tags <a name="patch-api-metadata-bulk-add"></a>
    ### Add Tags, ACLs and Legal Tags <a name="metadata-update-add"></a>
    In the "add" operation, the valid Tags, Legal Tags, and ACLs (Acl Viewers, Acl Owners) provided in the "value" field
    will be added to the property value in the "path" field. If we need to add tags ops.value should be colon separated
    ......@@ -575,7 +577,7 @@ curl --request PATCH \
    </details>
    ### Remove Tags, ACLs and Legal Tags <a name="patch-api-metadata-bulk-remove"></a>
    ### Remove Tags, ACLs and Legal Tags <a name="metadata-update-remove"></a>
    In the "remove" operation, the valid Tags, Legal Tags, and ACLs (Acl Viewers, Acl Owners) provided in the "value" field
    will be removed from the property value in the "path" field. When the given Tags, Legal Tags, or ACLs (Acl Viewers, Acl
    ......@@ -642,6 +644,96 @@ curl --request PATCH \
    > You can use Search service's query or query_with_cursor [apis](https://community.opengroup.org/osdu/platform/system/search-service/-/blob/master/docs/tutorial/SearchService.md) to search for records based on tags. Since tags is part of metadata, it is automatically indexed. This may not work if the kind is old (older than when the tags feature was introduced ~02/25/2021). You may need to re-index the kind with the [reindex](https://community.opengroup.org/osdu/platform/system/indexer-service/-/blob/master/docs/tutorial/IndexerService.md#reindex) api (with `force_clean=true`) from indexer service.
    ## Records patch api <a name="records-patch-api"></a>
    This API allows update of records data and/or metadata in batch. It takes an array of record ids (without version
    numbers) with a maximum number of 100, and updates properties specified in the operation path with value and operation type
    provided. Users need to specify the corresponding data partition id in the header as well.
    Users need to provide a list of recordIDs and a list of operations to be performed on each record. Each operation has `op`(operation type), `path`, and `value` in the field 'ops'. The currently supported operations are
    "replace", "add", and "remove". The supported properties for metadata update are `tags`, `acl/viewers`, `acl/owners`, `legal/legaltags`, `ancestry/parents` and `kind`.
    The supported properties for data update are `data` and `meta`. If `acl` is being updated, the user should be part of the groups that are being replaced/added/removed as ACL.
    User specifies the property they want to update in the `path` field, and new values should be provided in the `value` field.
    Note that this API supports PATCH operation in compliant to the [Patch RFC spec](https://www.rfc-editor.org/rfc/rfc6902). The behavior of the API reflects this accordingly.
    Records patch API has the following response codes:
    | Code | Description |
    |:-----|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    | 200 | The update operation succeeds fully, all records’ data and/or metadata get updated. | |
    | 206 | The update operation succeeds partially. Some records are not updated due to different reasons, including records not found or does not have permission to edit the records. |
    | 400 | The update operation fails when the remove operation makes Legal Tags or ACLs empty.| |
    Example request:
    <details><summary>curl</summary>
    ```
    curl --request PATCH \
    --url '/api/storage/v2/records' \
    --header 'accept: application/json' \
    --header 'authorization: Bearer <JWT>' \
    --header 'content-type: application/json-patch+json'\
    --header 'Data-Partition-Id: common'
    --data-raw ‘{
    "query": {
    "ids": [
    "tenant1:type:unique-identifier",
    "tenant2:type:unique-identifier",
    "tenant3:type:unique-identifier"
    ]
    },
    "ops": [
    {
    "op": "remove",
    "path": "/legal/legaltags/0"
    },
    {
    "op": "remove",
    "path": "/ancestry/parents"
    },
    {
    "op": "add",
    "path": "/acl/viewers/~",
    "value": "data.default.viewer1@opendes.enterprisedata.cloud.slb-ds.com"
    },
    {
    "op":"replace",
    "path":"/kind",
    "value":"newKind"
    },
    {
    "op":"add",
    "path":"/tags",
    "value":{
    "tag1":"value1",
    "tag2":"value2"
    }
    },
    {
    "op":"replace",
    "path":"/data/someProperty/targetProperty",
    "value": {
    "newValue": {
    "subProperty":"subValue"
    }
    }
    }
    ]
    }
    ```
    ### Differences compared to metadata update api <a name="patch-apis-diff"></a>
    | | Metadata Update API | Patch API |
    |-----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|
    | Header `Content-Type` | application/json | application/json-patch+json |
    | Supported Record properties | acl, tags, legaltags | acl, tags, legaltags, ancestry, kind, data, meta |
    | Supported operations | add, remove, replace | add, remove, replace |
    | `ops` field in payload | array of [PatchOperation](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/blob/master/src/main/java/org/opengroup/osdu/core/common/model/storage/PatchOperation.java) | [JsonPatch](https://www.rfc-editor.org/rfc/rfc6902) |
    | Maximum number of records | 500 | 100 |
    ## Using service accounts to access Storage APIs <a name="Service-accounts"></a>
    The Storage service relies on the Google native data access authorization mechanisms to provide access control on the records.
    Based on design decisions, when the Storage service caller is a federated user, no additional configuration is necessary, however if the API caller is a service account, a mandatory configuration is necessary as follows:
    ......
    0% Loading or .
    You are about to add 0 people to the discussion. Proceed with caution.
    Finish editing this message first!
    Please register or to comment