Adding headers to Put the file on Dataset Service
Context:
This issue goal is to put the Azure manifest json files into storage service. Currently header for this request is not considered in osdu-airflow-lib The call from _put_file_on_dataset_service
which calls the osdu_api is making the ingestion process fail.
<Error>
<Code>MissingRequiredHeader</Code>
<Message>An HTTP header that's mandatory for this request is not specified.
RequestId:bdf6dae2-701e-004f-2529-367fcb000000
Time:2023-02-01T10:37:20.2256774Z</Message>
<HeaderName>x-ms-blob-type</HeaderName>
</Error>
The "x-ms-blob-type" header is used in the Azure Blob storage service to specify the type of blob that is being uploaded. It ensures that the correct type of blob is being uploaded. The accepted values for the "x-ms-blob-type" header is "BlockBlob".
Call from osdu-airflow-lib:
put_result = dataset_dms_client.make_request(method=HttpMethod.PUT, url=signed_url, data=file_content, no_auth=True)
Proposal:
• The dataset service should be able to pass the headers while calling OSDU_API
cc: @Srinivasan_Narayanan @chad @valentin.gauthier @Yan_Sushchynski @nursheikh