CSV parser DAG on IBM R3 M19 environment is failing. Issue seems to be configuration and or resource allocation
When one looks at the DAG runs for CSV parser, there is not a single run that has been successful.
The log indicates that there is some issue in dict - undefined (may be setup configuration issue)
[2023-08-04 19:00:31,847] {standard_task_runner.py:77} INFO - Job 986: Subtask csv-parser
[2023-08-04 19:00:32,162] {logging_mixin.py:109} INFO - Running <TaskInstance: csv-parser-dag.csv-parser 2023-08-04T18:55:28.858482+00:00 [running]> on host ***-worker-0.***-worker.osdu.svc.cluster.local
[2023-08-04 19:00:32,194] {taskinstance.py:1463} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1165, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1246, in _prepare_and_execute_task_with_callbacks
self.render_templates(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1755, in render_templates
self.task.render_template_fields(context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 997, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1010, in _do_render_template_fields
rendered_content = self.render_template(content, context, jinja_env, seen_oids)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1061, in render_template
return [self.render_template(element, context, jinja_env) for element in content]
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1061, in <listcomp>
return [self.render_template(element, context, jinja_env) for element in content]
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1047, in render_template
return jinja_env.from_string(content).render(**context)
File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'correlationId'
[2023-08-04 19:00:32,197] {taskinstance.py:1506} INFO - Marking task as FAILED. dag_id=csv-parser-dag, task_id=csv-parser, execution_date=20230804T185528, start_date=20230804T190031, end_date=20230804T190032