Skip to content

Jhallal/dask update and optimization

Jeremie Hallal requested to merge jhallal/dask_update_and_optimization into master

Dask 2021.07.2 provide memory optimization.
There is also a new parrameter on read_parquet ("aggregate_files") that will help when we receive lots of small chunks https://docs.dask.org/en/latest/changelog.html

Merge request reports