IBM-CSV Ingestion-Platform Validation-Error File. The problem is also seen in Pre-ship environment
[2023-02-28 20:55:07,709] {taskinstance.py:1107} INFO - Executing <Task(KubernetesPodOperator): csv-parser> on 2023-02-28T20:50:03.419560+00:00 [2023-02-28 20:55:07,714] {standard_task_runner.py:52} INFO - Started process 2868 to run task [2023-02-28 20:55:07,720] {standard_task_runner.py:76} INFO - Running: ['', 'tasks', 'run', 'csv-parser-dag', 'csv-parser', '2023-02-28T20:50:03.419560+00:00', '--job-id', '58359', '--pool', 'default_pool', '--raw', '--subdir', 'DAGS_FOLDER/csv_ingestion_all_steps.py', '--cfg-path', '/tmp/tmpuolct0ay', '--error-file', '/tmp/tmpeb5k26k9'] [2023-02-28 20:55:07,721] {standard_task_runner.py:77} INFO - Job 58359: Subtask csv-parser [2023-02-28 20:55:07,780] {logging_mixin.py:104} INFO - Running <TaskInstance: csv-parser-dag.csv-parser 2023-02-28T20:50:03.419560+00:00 [running]> on host -worker-0.-worker.osdu-.svc.cluster.local [2023-02-28 20:55:07,846] {taskinstance.py:1501} ERROR - Task failed with exception Traceback (most recent call last): File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1157, in _run_raw_task self._prepare_and_execute_task_with_callbacks(context, task) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1294, in _prepare_and_execute_task_with_callbacks self.render_templates(context=context) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1795, in render_templates self.task.render_template_fields(context) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1003, in render_template_fields self._do_render_template_fields(self, self.template_fields, context, jinja_env, set()) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1016, in _do_render_template_fields rendered_content = self.render_template(content, context, jinja_env, seen_oids) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1067, in render_template return [self.render_template(element, context, jinja_env) for element in content] File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1067, in return [self.render_template(element, context, jinja_env) for element in content] File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1053, in render_template return jinja_env.from_string(content).render(**context) File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render self.environment.handle_exception() File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception reraise(*rewrite_traceback_stack(source=source)) File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise raise value.with_traceback(tb) File "", line 1, in top-level template code jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'correlationId' [2023-02-28 20:55:07,851] {taskinstance.py:1544} INFO - Marking task as FAILED. dag_id=csv-parser-dag, task_id=csv-parser, execu