Infrastructure Submissions:
- [YES] Have you added an explanation of what your changes do and why you'd like us to include them?
- [YES/NO] I have updated the documentation accordingly.
- [YES/NO/NA] I have added tests to cover my changes.
- [YES] All new and existing tests passed.
- [YES] I have formatted the terraform code. (
terraform fmt -recursive && go fmt ./...)
Current Behavior or Linked Issues
The file-staging-area Blob Container is used by File Service for Generating Signed URLs for Upload operations. Once the file is uploaded, the client uses File Service's metadata API which will internally copy the blob contents from the file-staging-area to file-persistent-area and update the blob's metadata to Storage Service.
This results in lot of temporary files left over in the Staging area. Removing the files periodically will better the COGS
Feature Change - Support auto delete of files on file-staging-area #177 (closed)
Does this introduce a breaking change?
- [NO]
Other information
After this change, all the baseBlobs in the staging area will get deleted after the specified number of retention days.
The number of retention days is a configurable number and this feature is enabled behind a feature flag.
To view the change, a lifecycle management policy gets added in the azure portal corresponding to the specific storage account
pipeline plans:
Disabled flag https://dev.azure.com/osdu-demo/OSDU_Rx/_build/results?buildId=141386&view=results
Enabled flag: https://dev.azure.com/osdu-demo/OSDU_Rx/_build/results?buildId=141377&view=results