Skip to content

Adding storage lifecycle management policy

Nidhi Jain requested to merge feature/storage-mgmt-policy into master

Infrastructure Submissions:

  • [YES] Have you added an explanation of what your changes do and why you'd like us to include them?
  • [YES/NO] I have updated the documentation accordingly.
  • [YES/NO/NA] I have added tests to cover my changes.
  • [YES] All new and existing tests passed.
  • [YES] I have formatted the terraform code. (terraform fmt -recursive && go fmt ./...)

Current Behavior or Linked Issues

The file-staging-area Blob Container is used by File Service for Generating Signed URLs for Upload operations. Once the file is uploaded, the client uses File Service's metadata API which will internally copy the blob contents from the file-staging-area to file-persistent-area and update the blob's metadata to Storage Service.

This results in lot of temporary files left over in the Staging area. Removing the files periodically will better the COGS

Feature Change - Support auto delete of files on file-staging-area #177 (closed)

Does this introduce a breaking change?

  • [NO]

Other information

After this change, all the baseBlobs in the staging area will get deleted after the specified number of retention days. The number of retention days is a configurable number and this feature is enabled behind a feature flag. To view the change, a lifecycle management policy gets added in the azure portal corresponding to the specific storage account image

pipeline plans:

Disabled flag https://dev.azure.com/osdu-demo/OSDU_Rx/_build/results?buildId=141386&view=results

Enabled flag: https://dev.azure.com/osdu-demo/OSDU_Rx/_build/results?buildId=141377&view=results

Edited by Nidhi Jain

Merge request reports