Skip to content

feat: [For MSFT Team Review]: bulk delete filters API

[Internal PR, target branch is not master]

Updates description?

Third part of the implementation of advanced filters for #117.

This PR includes:

  • Advanced filters in Bulk Delete operation

Still missing, will be provided in follow-up PRs:

  • Support for or and not operators

How to test

Run the application and the deletion sidecar locally (see PR 455 in MSFT internal repo for instructions on running the sidecar).

Use the Swagger UI to create datasets and delete them selectively using filters. Below is a sample end-to-end scenario.

Create a first dataset

Use the POST /dataset/tenant/{tenantid}/subproject/{subprojectid}/dataset/{datasetid} endpoint to create a dataset, with the following parameters:

tenantId: read
subprojectId: subproject123
datasetId: bulkdeltest01
path: /bulkdeltest

and the following body:

{
    "gtags": [
      "bulktesttag01"
    ]
  }

Use the PUT /dataset/tenant/{tenantid}/subproject/{subprojectid}/dataset/{datasetid}/unlock endpoint to remove the write lock, with the following parameters:

tenantId: read
subprojectId: subproject123
datasetId: bulkdeltest01
path: /bulkdeltest

Create a second dataset at the same path

Repeat the two operations above, replacing every occurrence of 01 with 02 (in datasetId and the tag name).

List datasets

Use the GET /dataset/tenant/{tenantid}/subproject/{subprojectid} endpoint to list datasets, with the following parameters:

select: [name]
tenantId: read
subprojectId: subproject123

Ensure that the bulkdeltest01 and bulkdeltest02 datasets are listed.

Run bulk delete

Use the PUT /operation/bulk-delete operation, with the following parameter:

path: sd://read/subproject123/bulkdeltest

and the following body:

{
  "filter": {
    "and": [
      {
        "property": "gtags",
        "operator": "CONTAINS",
        "value": "bulktesttag01"
      },
      {
        "property": "name",
        "operator": "LIKE",
        "value": "bulkdeltest%"
      }
    ]
  }
}

Check the console of the deletion sidecar for success messages.

List datasets

Perform the call to the list datasets endpoint again. Ensure that only the bulkdeltest02 dataset is listed.

Cleanup

Use the bulk delete endpoint again, this time with bulktesttag02 as tag filter value. Ensure that the list datasets endpoint does not show any datasets named bulkdeltest*.

Edited by Alexandre Gattiker

Merge request reports