quantized model : all convolutions are in fp32

[etienne87]

i am running this example : https://github.com/openvinotoolkit/nncf/blob/develop/examples/post_training_quantization/torch/ssd300_vgg16/main.py
the int8 model runs faster:

[1/7] Save FP32 model: /home/eperot/skeleton-recall/mobilenet_v2_fp32.xml
Model graph (xml):   0.133 Mb
Model weights (bin): 8.467 Mb
Model size:          8.600 Mb
[2/7] Save INT8 model: /home/eperot/skeleton-recall/mobilenet_v2_int8.xml
Model graph (xml):   0.380 Mb
Model weights (bin): 2.295 Mb
Model size:          2.675 Mb
[3/7] Benchmark FP32 model:
[ INFO ] Count:            237824 iterations
[ INFO ] Duration:         15007.30 ms
[ INFO ] Latency:
[ INFO ]    Median:        3.56 ms
[ INFO ]    Average:       3.91 ms
[ INFO ]    Min:           2.96 ms
[ INFO ]    Max:           34.62 ms
[ INFO ] Throughput:   15847.22 FPS
[4/7] Benchmark INT8 model:
[ INFO ] Count:            369856 iterations
[ INFO ] Duration:         15005.44 ms
[ INFO ] Latency:
[ INFO ]    Median:        2.21 ms
[ INFO ]    Average:       2.47 ms
[ INFO ]    Min:           1.96 ms
[ INFO ]    Max:           23.25 ms
[ INFO ] Throughput:   24648.13 FPS
[5/7] Validate OpenVINO FP32 model:
Validating ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:14
Accuracy @ top1: 0.986
[6/7] Validate OpenVINO INT8 model:
Validating ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:13
Accuracy @ top1: 0.985
[7/7] Report:
Accuracy drop: 0.001
Model compression rate: 3.215

but when i read the xml file and check at precision on conv i see no "int8"

i see sections like this only:

<layer id="382" name="Multiply_35620" type="Convolution" version="opset1">
			<data strides="1, 1" dilations="1, 1" pads_begin="0, 0" pads_end="0, 0" auto_pad="explicit" />
			<input>
				<port id="0" precision="FP32">
					<dim>-1</dim>
					<dim>384</dim>
					<dim>-1</dim>
					<dim>-1</dim>
				</port>
				<port id="1" precision="FP32">
					<dim>64</dim>
					<dim>384</dim>
					<dim>1</dim>
					<dim>1</dim>
				</port>
			</input>
			<output>
				<port id="2" precision="FP32">
					<dim>-1</dim>
					<dim>64</dim>
					<dim>-1</dim>
					<dim>-1</dim>
				</port>
			</output>
		</layer>

mobilenet_v2_int8_xml.json

[AlexKoff88]

Hi @etienne87, thanks for your interest.

You should search for "i8" in the .xml file.
You need more details how quantization is represented and used in the OpenVINO IR, please refer to the:

• https://docs.openvino.ai/2024/documentation/openvino-ir-format/operation-sets/operation-specs/quantization/fake-quantize-1.html
• https://docs.openvino.ai/2024/documentation/openvino-extensibility/openvino-plugin-library/advanced-guides/low-precision-transformations.html

and other relevant documents.