Manifest ingestion does not show any updates in airflow when backslash character used in json body
Description: Using manifest ingestion (DAG) workflow service, when user insert backslash \ into json body manifest the workflow run stucks in SUBMITTED status. There is also no trace of the runID running in the Airflow log.
Steps to reproduce:
a) Insert the body json into DAG worklow body. With_Backslash_BodyData.json
b) When submit the body json into base_url/api/workflow/v1/workflow/Osdu_ingest/workflowRun the workflow is submitted succesfully with the following response:
{ "workflowId": "dev:Osdu_ingest", "runId": "4327f575-e7b3-490f-a1ee-b1e2e950c2a4", "startTimeStamp": 1627041278115, "status": "submitted", "submittedBy": "naufal.noori@katalystdm.com" }
c) After a while, check DAG run status and the workflow still showing the run is in submitted status. And no trace of the run ID in the Airflow log (This follow up check was done after 24 hours):
Endpoint: base_url/api/workflow/v1/workflow/Osdu_ingest/workflowRun/4327f575-e7b3-490f-a1ee-b1e2e950c2a4
Response:
{ "workflowId": "dev:Osdu_ingest", "runId": "4327f575-e7b3-490f-a1ee-b1e2e950c2a4", "startTimeStamp": 1627041278115, "status": "submitted", "submittedBy": "naufal.noori@katalystdm.com" }
d) When a second trial run was conducted by replacing \ char with empty char, the workflow run was running perfectly and shows trace of running in Airflow log. With_NO_Backslash_BodyData.json
Expectation: We are expecting that the workflow run to failed our request with clear and meaningful error message i.e. Request is failed. There is non-allowed special characters in line #something to line #something in your json body.
Reason It will be a confusion for users to have a run successfully submitted but stuck in the process without any log trace whatsoever.
cc @debasisc