FP8 quantization failed for whisper_base model

[TolyaTalamanov]

Environment

python -m venv env
source env/bin/activate
pip install onnx==1.17.0
pip install nncf==2.14
pip install git+https://github.com/huggingface/optimum-intel.git@2590794f5c4372b2849e2b06744a0ae651d66642#egg=optimum-intel
pip install --pre openvino==2025.1.0.dev20250116 --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

Run:

optimum-cli export openvino -m openai/whisper-base --disable-stateful --quant-mode f8e4m3 --sym whisper-base-fp8e4m3-sym

Error:

Mismatched elements: 103730 / 103730 (100.0%)
Greatest absolute difference: 16.697378158569336 at index (1, 0, 50258) (up to 1e-05 allowed)
Greatest relative difference: 1.6641814239832255 at index (1, 0, 50258) (up to 1e-05 allowed)
  _check_trace(
Tokenizer won't be converted.
Traceback (most recent call last):
  File "/home/atalaman/workspace/npuw-development/whisper/models/env/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
  File "/home/atalaman/workspace/npuw-development/whisper/models/env/lib/python3.10/site-packages/optimum/commands/optimum_cli.py", line 208, in main
    service.run()
  File "/home/atalaman/workspace/npuw-development/whisper/models/env/lib/python3.10/site-packages/optimum/commands/export/openvino.py", line 454, in run
    main_export(
  File "/home/atalaman/workspace/npuw-development/whisper/models/env/lib/python3.10/site-packages/optimum/exporters/openvino/__main__.py", line 490, in main_export
    _weight_only_quantization(submodel, quantization_config)
  File "/home/atalaman/workspace/npuw-development/whisper/models/env/lib/python3.10/site-packages/optimum/intel/openvino/quantization.py", line 1031, in _weight_only_quantization
    if isinstance(config.sensitivity_metric, str):
AttributeError: 'OVQuantizationConfig' object has no attribute 'sensitivity_metric'

[nikita-savelyevv]

Thanks for reporting. --dataset librispeech argument should be provided for full quantization. I understand though that the error is misleading. I've added raising of an explicit error in such case: #1120 .