Skip to content

Airflow pgbouncer changes

Vineeth Guna [Microsoft] requested to merge airflow_pgbouncer_changes into master

Infrastructure Submissions:

  • [YES] Have you added an explanation of what your changes do and why you'd like us to include them?
  • [NO] I have updated the documentation accordingly
  • [NA] I have added tests to cover my changes.
  • [YES] All new and existing tests passed.
  • [YES] I have formatted the terraform code. (terraform fmt -recursive && go fmt ./...)

Current Behavior or Linked Issues

The changes in the MR will effect the way airflow components (webserver, scheduler, workers) will connect to the postgresql database.

With this change the airflow components will connect to pgbouncer (a lightweight connection pooling proxy for postgresql) and internally pgbouncer connects to Azure Postgresql Server

This change reduces Postgresql bottlenecks observed during performance and scalability testing

Does this introduce a breaking change?

  • NO

Other information

Edited by Vineeth Guna [Microsoft]

Merge request reports