DeepSpeed v0.6.0

DeepSpeed v0.6.0

Release notes

New features

• Advancing MoE inference and training to power next-generation AI scale
  • MoE inference
  • PR-MoE model support
• AMD support (#1430)
• Various ZeRO Stage3 Optimizations + Improvements (#1453)

Special thanks to our contributors in this release

@stas00, @jithunnair-amd, @rraminen, @jeffdaily, @okakarpa, @jfc4050, @raamjad, @aphedges, @SeanNaren, @liamcli, @andriyor, @manuelciosici

v0.5.10: Patch release

[ZeRO] Default disable elastic ckpt in stage 1+2 and reduce CPU memor…

v0.5.9: Patch release

Various small documentation text improvements (#1665)

Co-authored-by: Jeff Rasley <[email protected]>

v0.5.8: Patch release

Remove unused import of ssl.OP_ENABLE_MIDDLEBOX_COMPAT (#1601)

v0.5.7: Patch release

Several fixes for our read-the-docs build (#1579)

v0.5.6: Patch release

Fix 1bit extra issue (#1542)

v0.5.5: Patch release

Prevent creation of local temp directory (#1494)

Co-authored-by: Olatunji Ruwase <[email protected]>
Co-authored-by: Jeff Rasley <[email protected]>

v0.5.4: Patch release

Fix the workspace allocation for the transformer kernel (#1397)

* fix the workspace allocation for the transformer kernel

* change layer-id type & rm one unit test due to OOM

v0.5.3: Patch release

[zero_to_fp32] adapt to 4-bytes alignment in z2 (#1372)

Co-authored-by: Olatunji Ruwase <[email protected]>

v0.5.2: Patch release

Update setup.py (#1361)

updated classifiers