Skip to content

v1.3.0: Knowledge distillation and one-shot optimization support

Compare
Choose a tag to compare
@echarlaix echarlaix released this 05 Aug 15:26
· 969 commits to main since this release

Knowledge distillation

Knowledge distillation was introduced in #8. To perform distillation, an IncDistiller must be instantiated with the appropriate configuration.

One-shot optimization

The possibility to combine compression techniques such as pruning, knowledge distillation and quantization aware training in one-shot during training was introduced (#7). One-shot optimization is set by default, but can be cancelled by setting the one_shot_optimization parameter to False when instantiating the IncOptimizer.

Seq2Seq models support

Both quantization and pruning can now be applied on Seq2Seq models (#14)