Configure retry for OSDU Ingest
PBI: https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/\_workitems/edit/15576 Setting up retries for OSDU ingest
============== Tested the retries on a test instance and verified the functionality is working. Test Dags:
- https://adf-4q2m464plpuwq-airflowir-2707.westeurope.airflow.svc.datafactory.azure.com/tree?dag_id=sstest_Osdu_ingest
- https://adf-4q2m464plpuwq-airflowir-2707.westeurope.airflow.svc.datafactory.azure.com/tree?dag_id=sstest_csv-parser
- https://adf-4q2m464plpuwq-airflowir-2707.westeurope.airflow.svc.datafactory.azure.com/log?dag_id=sstest0301_osdu-ingest-r3&task_id=process_manifest_task_1&execution_date=2023-01-03T08%3A37%3A16.035579%2B00%3A00
=====================================
Template -
original osdu-ingest-r3 file:
Generated osdu-ingest-r3 file after template replaces placeholders:
Tested in airflow portal and retries were successful:
