Upgrade python-linters to support Python 3.11 with Airflow 2.10.5
Background
- Updates to the latest Airflow and Python version (compatible with M25) is required to ensure that we are using the latest image and packages for our pipeline
Changes
- Python Image Upgrade:
python:3.8.12-slim->python:3.11-slim - Airflow Image Upgrade:
apache/airflow:2.1.2-python3.8->apache/airflow:2.10.5-python3.11 - Change package installation of
pylintpylint-exitto always pull the latest package compatible - Utilizing
pylint-exitas to generate summary of the linting and ensuring that the exit code is clean and easily understood
Results
- pylint (#3634292) · Jobs · OSDU / OSDU Data Platform / Data Flow / Data Ingestion / osdu-airflow-lib · GitLab
- Exit code and summaries are generated like below
Your code has been rated at 9.18/10
The following messages were raised:
- error message issued
- warning message issued
- refactor message issued
- convention message issued
No fatal messages detected. Exiting gracefully...
$ exit ${EXIT_CODE}
Cleaning up project directory and file based variables
Job succeeded
Edited by Teo Sheng Pu