Fix default num samples #1129
Fix default num samples #1129
Conversation
|
@AlexKoff88, @nikita-savelyevv, could you review this PR, please? Thanks. |
| @@ -1083,7 +1083,7 @@ def _full_quantization( | |||
| quantized_model = nncf.quantize( | |||
| model, | |||
| calibration_dataset, | |||
| subset_size=quantization_config.num_samples, | |||
| subset_size=quantization_config.num_samples if quantization_config.num_samples else 128, | |||
There was a problem hiding this comment.
| subset_size=quantization_config.num_samples if quantization_config.num_samples else 128, | |
| subset_size=quantization_config.num_samples if quantization_config.num_samples else 300, |
For nncf.quantize() the default value is currently 300.
There was a problem hiding this comment.
@AlexKoff88, could you please clarify, which numbers should be as the default.
There was a problem hiding this comment.
300 is a default value for CNNs. We noticed that for LLMs it can be less. I think it is ok to use 128 and keep it aligned with weight compression.
tests/openvino/test_exporters_cli.py
Outdated
| ( | ||
| "text-generation", | ||
| "llama", | ||
| "int8", | ||
| "--dataset wikitext2 --trust-remote-code", | ||
| (13,), | ||
| (16,), | ||
| ), |
There was a problem hiding this comment.
Could you remove --num-samples 1 from the previous test instead of adding this one? This will have less impact on testing time and maintain the same coverage.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
What does this PR do?
Fixes 161201
Before submitting