ptq_static

Step-by-Step

This example load an image classification model exported from PyTorch and confirm its accuracy and speed based on ILSVR2012 validation Imagenet dataset. You need to download this dataset yourself.

Prerequisite

1. Environment

pip install neural-compressor
pip install -r requirements.txt

Note: Validated ONNX Runtime Version.

2. Prepare Model

python prepare_model.py --input_model='resnet50_v1.pb' --output_model='resnet50_v1.onnx'

Note: For now, use "onnx==1.14.1" in this step in case you get an error ValueError: Could not infer attribute explicit_paddings type from empty iterator. Refer to this link for more details of this error.

3. Prepare Dataset

Download dataset ILSVR2012 validation Imagenet dataset.

Download label:

wget http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz
tar -xvzf caffe_ilsvrc12.tar.gz val.txt

Run

1. Quantization

Quantize model with QLinearOps:

bash run_quant.sh --input_model=path/to/model \  # model path as *.onnx
                   --dataset_location=/path/to/imagenet \
                   --label_path=/path/to/val.txt \
                   --output_model=path/to/save

Quantize model with QDQ mode:

bash run_quant.sh --input_model=path/to/model \  # model path as *.onnx
                   --dataset_location=/path/to/imagenet \
                   --label_path=/path/to/val.txt \
                   --output_model=path/to/save \
                   --quant_format=QDQ

2. Benchmark

bash run_benchmark.sh --input_model=path/to/model \  # model path as *.onnx
                      --dataset_location=/path/to/imagenet \
                      --label_path=/path/to/val.txt \
                      --batch_size=batch_size \
                      --mode=performance # or accuracy