feat: [For MSFT Team Review]: bulk delete filters API
[Internal PR, target branch is not master
]
Updates description?
Third part of the implementation of advanced filters for #117 (closed).
This PR includes:
- Advanced filters in Bulk Delete operation
Still missing, will be provided in follow-up PRs:
- Support for
or
andnot
operators
How to test
Run the application and the deletion sidecar locally (see PR 455 in MSFT internal repo for instructions on running the sidecar).
Use the Swagger UI to create datasets and delete them selectively using filters. Below is a sample end-to-end scenario.
Create a first dataset
Use the POST /dataset/tenant/{tenantid}/subproject/{subprojectid}/dataset/{datasetid}
endpoint to create a dataset, with the following parameters:
tenantId: read
subprojectId: subproject123
datasetId: bulkdeltest01
path: /bulkdeltest
and the following body:
{
"gtags": [
"bulktesttag01"
]
}
Use the PUT /dataset/tenant/{tenantid}/subproject/{subprojectid}/dataset/{datasetid}/unlock
endpoint to remove the write lock, with the following parameters:
tenantId: read
subprojectId: subproject123
datasetId: bulkdeltest01
path: /bulkdeltest
Create a second dataset at the same path
Repeat the two operations above, replacing every occurrence of 01
with 02
(in datasetId
and the tag name).
List datasets
Use the GET /dataset/tenant/{tenantid}/subproject/{subprojectid}
endpoint to list datasets, with the following parameters:
select: [name]
tenantId: read
subprojectId: subproject123
Ensure that the bulkdeltest01
and bulkdeltest02
datasets are listed.
Run bulk delete
Use the PUT /operation/bulk-delete
operation, with the following parameter:
path: sd://read/subproject123/bulkdeltest
and the following body:
{
"filter": {
"and": [
{
"property": "gtags",
"operator": "CONTAINS",
"value": "bulktesttag01"
},
{
"property": "name",
"operator": "LIKE",
"value": "bulkdeltest%"
}
]
}
}
Check the console of the deletion sidecar for success messages.
List datasets
Perform the call to the list datasets endpoint again. Ensure that only the bulkdeltest02
dataset is listed.
Cleanup
Use the bulk delete endpoint again, this time with bulktesttag02
as tag filter value. Ensure that the list datasets endpoint does not show any datasets named bulkdeltest*
.