Skip to content
Snippets Groups Projects
Commit d8ad7c33 authored by Cyril Monmouton's avatar Cyril Monmouton
Browse files

Remove Dask scatter method because Dask is not used anymore to save catalog

parent c09d09c8
No related branches found
No related tags found
1 merge request!460Improve statibility of fetching of bulk catalog by not using Dask workers
......@@ -470,8 +470,7 @@ class DaskBulkStorage:
self._fill_catalog_columns_info(catalog, chunk_metas, bulk_id)
)
fcatalog = await self.client.scatter(catalog)
await save_bulk_catalog(self._fs, commit_path, await fcatalog)
await save_bulk_catalog(self._fs, commit_path, catalog)
trace_attributes_root_span({
'catalog-row-count': catalog.nb_rows,
'catalog-col-count': catalog.all_columns_count
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment