Migrate Postgres DB for airflow
- Airflow is currently using postgresql single server with postgres version 10.
- Azure Database for PostgreSQL Single Server will be retired in March 2025.
- Migration to flexible server is necessary to use PostgreSQL versions > 10
- Refer Azure postgresql migration tool
- Pertaining to above reasons upgrading airflow db to flexible server with postgresql 14 version is recommended.
------------------------------------------------------------------------------
- Currently, Apache airflow uses Pgbouncer to do connection pooling.
- It is using outdated
mcr.microsoft.com/azure-oss-db-tools/pgbouncer-sidecar:pgbouncerv1.13
- Since
Postgresql flexible server
natively supports pgbouncer, https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-pgbouncer It is advisable to use it. - It would require helm changes to disable pgbouncer in helm charts and modify port configs and few other connection config changes.
Related to #324 (closed)
Edited by saketh somaraju [EPAM]