2024.11.2

Changes

• Bump JamesIves/github-pages-deploy-action from 4.6.8 to 4.6.9 @dependabot (#8931)
• Remove alias resolving to fix queuing @phofl (#8933)

See the Changelog for more information.

2024.11.1

Changes

• Skip collecting coverage for CLI tests @fjetter (#8930)

See the Changelog for more information.

2024.11.0

Changes

• Remove redundant methods in P2PBarrierTask @fjetter (#8924)
• fix skipif condition for test_tell_workers_when_peers_have_left @fjetter (#8929)
• Ensure ConnectionPool closes even if network stack swallows cancellation @fjetter (#8928)
• Fix flaky test_server_comms_mark_active_handlers @fjetter (#8927)
• Make assumption in P2P's barrier mechanism explicit @hendrikmakait (#8926)
• Adjust timeouts in jupyter cli test @fjetter (#8925)
• Add stimulus_id to SchedulerPlugin.update_graph hook @hendrikmakait (#8923)
• Reduce P2P transfer task overhead @hendrikmakait (#8912)
• Disable profiler on python 3.11 @fjetter (#8916)
• Fix test_restarting_does_not_deadlock @fjetter (#8849)
• Adjust popen timeouts for testing @fjetter (#8848)
• Add retry to shuffle broadcast @fjetter (#8900)
• Fix test_shuffle_with_array_conversion @fjetter (#8909)
• Refactor some tests @fjetter (#8908)
• Reflect graduation of dask-expr from dask-contrib to dask @hendrikmakait (#8911)
• Skip test_tell_workers_when_peers_have_left on py3.10 @fjetter (#8910)
• Internal cleanup of P2P code @hendrikmakait (#8907)
• Use Task class instead of tuple @fjetter (#8797)
• Increase connect timeout for test_tell_workers_when_peers_have_left @fjetter (#8906)
• Remove dispatching in TaskCollection @fjetter (#8903)
• Deduplicate scheduler requests in P2P @hendrikmakait (#8899)
• Add configurations for rootish taskgroup threshold @phofl (#8898)

See the Changelog for more information.

2024.10.0

Changes

• Make P2P more configurable @hendrikmakait (#8469)
• Fit Dashboard worker table to page width @jacobtomlinson (#8897)
• Raise helpful error when using the wrong plugin base classes @jacobtomlinson (#8893)
• Fix doc build @phofl (#8892)
• Fix url escaping on exceptions dashboard for non-string keys @phofl (#8891)
• Explicitly list setuptools as a build dependency in conda recipe @charlesbluca (#8890)
• Avoid spurious error log in Adaptive @hendrikmakait (#8887)
• Add meaningful error for out of disk exception during write @hendrikmakait (#8886)
• Bump JamesIves/github-pages-deploy-action from 4.6.4 to 4.6.8 @dependabot (#8880)
• Switch from mambaforge to miniforge in CI @jrbourbeau (#8881)

See the Changelog for more information.

2024.9.1

Changes

• Don't stop Adaptive on error @hendrikmakait (#8871)
• Update gpuCI RAPIDS_VER to 24.12 @github-actions (#8879)
• Don't consider scheduler idle while executing Scheduler.update_graph @hendrikmakait (#8877)
• Bump jacobtomlinson/gha-anaconda-package-version from 0.1.3 to 0.1.4 @dependabot (#8878)
• Support P2P rechunking datetime arrays @jrbourbeau (#8875)

See the Changelog for more information.

2024.9.0

Changes

• Homogeneously schedule P2P's unpack tasks @hendrikmakait (#8873)
• Work/fix firewall for localhost @maldag (#8868)
• Bump bokeh minimum version to 3.1.0 @jrbourbeau (#8861)
• Use new tokenize module @jrbourbeau (#8858)
• Point to user code with idempotent plugin warning @jrbourbeau (#8856)
• Fix test nanny timeout @fjetter (#8847)
• Bump JamesIves/github-pages-deploy-action from 4.5.0 to 4.6.4 @dependabot (#8853)
• Speed up Client.map by computing token only once for func and kwargs @fjetter (#8855)
• Update precommit @fjetter (#8852)

See the Changelog for more information.

2024.8.2

Changes

• Avoid capturing code of xdist @fjetter (#8846)
• Reduce memory footprint of culling P2P rechunking @hendrikmakait (#8845)
• Add tests for choosing default rechunking method @hendrikmakait (#8843)
• Increase visibility of GPU CI updates @charlesbluca (#8841)
• Bump test_pause_while_idle timeout @fjetter (#8844)
• Concatenate small input chunks before P2P rechunking @hendrikmakait (#8832)
• Remove dump cluster from gen_cluster @fjetter (#8823)
• Bump numpy>=1.24 and pyarrow>=14.0.1 minimum versions @jrbourbeau (#8837)
• Fix PipInstall plugin on Worker @hendrikmakait (#8839)
• Remove more Python 3.10 compatibility code @jrbourbeau (#8824)
• Use task-based rechunking to prechunk along partial boundaries @hendrikmakait (#8831)
• Ensure client_desires_keys does not corrupt Scheduler state @fjetter (#8827)
• Bump minimum cloudpickle to 3 @jrbourbeau (#8836)

See the Changelog for more information.

2024.8.1

Changes

• Reduce frequency of unmanaged memory use warning @phofl (#8834)
• Update gpuCI RAPIDS_VER to 24.10 @github-actions (#8786)
• Avoid RuntimeError: dictionary changed size during iteration in Server._shift_counters() @hendrikmakait (#8828)
• Improve concurrent close for scheduler @hendrikmakait (#8829)
• MINOR: Extract truncation logic out of partial concatenation in P2P rechunking @hendrikmakait (#8826)
• Avoid excessive attribute access overhead for remove_from_task_prefix_count @fjetter (#8821)
• Avoid key validation if validation is disabled @fjetter (#8822)
• Log worker\_client event @jrbourbeau (#8819)
• Drop support for Python 3.9 @phofl (#8793)

See the Changelog for more information.

2024.8.0

Changes

• Run graph normalisation after dask order @phofl (#8818)
• Update large graph size warning to remove scatter recommendation @phofl (#8815)
• Fail tasks exceeding no-workers-timeout @hendrikmakait (#8806)
• Fix exception handling for NannyPlugin.setup and NannyPlugin.teardown @hendrikmakait (#8811)
• Fix exception handling for WorkerPlugin.setup and WorkerPlugin.teardown @hendrikmakait (#8810)
• Fix typo in transitions event logging @alex-rakowski (#8812)
• Fix if-else for send_recv_from_rpc @phofl (#8809)
• Ensure that adaptive only stops once @hendrikmakait (#8807)
• Reduce noise from GC-related logging @hendrikmakait (#8804)
• Remove unused delete_interval and synchronize_worker_interval from Scheduler @hendrikmakait (#8801)
• Change log level for Compute Failed log message @phofl (#8802)
• Add Prometheus metric for time spent on GC @hendrikmakait (#8803)
• Add Prometheus metrics for dask_scheduler_workers_{added|removed}_total @hendrikmakait (#8798)
• Add log event for worker-ttl-timeout @hendrikmakait (#8800)
• Add Prometheus metrics for dask_scheduler_client_connections_{added|removed}_total @hendrikmakait (#8799)
• Fix restarting workers with PackageInstall plugin @hendrikmakait (#8794)
• Make stealing more robust @hendrikmakait (#8788)
• leave a warning about future instantiation @fjetter (#8782)

See the Changelog for more information.

2024.7.1

Changes

• Ensure Locks always register with scheduler @fjetter (#8781)
• Temporarily pin setuptools \< 71 @jrbourbeau (#8785)
• Restore len() on TaskPrefix @hendrikmakait (#8783)
• Avoid false positives for p2p-failed log event @hendrikmakait (#8777)
• Expose paused and retired workers separately in prometheus @phofl (#8613)
• Creating transitions-failures log event @alex-rakowski (#8776)
• Implement HLG layer for P2P rechunking @hendrikmakait (#8751)
• Ensure Lock will not lock up in case of worker failures @fjetter (#8770)
• Add another test for a possible deadlock scenario caused by #8703 @hendrikmakait (#8769)
• Raise an error if compute on persisted collection with released futures @fjetter (#8764)
• Re-raise P2PConsistencyError from failed P2P tasks. @hendrikmakait (#8748)
• Robuster faster tests memory sampler @fjetter (#8758)
• fix scheduler_bokeh::test_shuffling @fjetter (#8766)
• increase timeouts for pubsub::test_client_worker @fjetter (#8765)
• Factor out async taskgroup @fjetter (#8756)
• Don't sort keys lexicographically in worker table @fjetter (#8753)
• Use functools.cache instead of functools.lru_cache for url_escape & _expects_comm @jonded94 (#8762)
• Robuster deeply nested structures @fjetter (#8730)
• Adding HLG to MAP @alex-rakowski (#8740)
• Add close worker button to worker info page @jrbourbeau (#8742)

See the Changelog for more information.