[Ingestion] End to end ingestion does not work
Ingestion end to end does not work. The expectation is calling
curl --location --request POST 'https://{path}/submit'
--header 'Authorization: Bearer {token}'
--header 'Partition-Id: {assigned DELFI partition ID}'
--header 'Content-Type: application/json'
--data-raw '{
"FileID": "c26c7656-8c50-4147-b51f-c7a449af33f3",
"DataType": "opaque"
}'
end-point should result in a successful ingestion of the file denoted by FileID. Of course the file denoted by FileID has been uploaded using the signed url returned in the response of FileService /getLocation api.
This does not happen as the DAG create_records.py fails.
Appreciate to have a dialogue on this issue and next steps. I will provide more information as a conversation thread on this issue starts.