Airflow pgbouncer changes
Infrastructure Submissions:
- [YES] Have you added an explanation of what your changes do and why you'd like us to include them?
- [NO] I have updated the documentation accordingly
- [NA] I have added tests to cover my changes.
- [YES] All new and existing tests passed.
- [YES] I have formatted the terraform code. (
terraform fmt -recursive && go fmt ./...)
Current Behavior or Linked Issues
The changes in the MR will effect the way airflow components (webserver, scheduler, workers) will connect to the postgresql database.
With this change the airflow components will connect to pgbouncer (a lightweight connection pooling proxy for postgresql) and internally pgbouncer connects to Azure Postgresql Server
This change reduces Postgresql bottlenecks observed during performance and scalability testing
Does this introduce a breaking change?
- NO