fix: split batchclient delete request into multiple

Type of change

• Bug Fix
• Feature
• Pipeline
• Test
• Documentation

Does this introduce a change in the core logic?

• No
• Yes

Does this introduce a change in the cloud provider implementation, if so which cloud?

• AWS
• Anthos
• Azure
• GCP
• IBM

Does this follow conventional commits spec?

• No
• Yes

Have you set the target Milestone?

• No
• Yes

Have you set the no-detached-pipeline label?

• No
• Yes

Updates description?

MR splits the batchclient's delete request into multiple. The batchClient from Azure SDK has a limit of 256 changes per request, if a dataset has more than 256 blobs it will have an error. The cosmosDB metadata is deleted before contacting storage to delete the blobs, so now the blobs are orphaned.

For large datasets should use the builk-delete operation endpoint