This folder contains five notebooks that show how to train, optimize, quantize and show live inference on a MONAI segmentation model with PyTorch Lightning and OpenVINO:
1. Data Preparation for 2D Segmentation of 3D Medical Data
2. Train a 2D-UNet Medical Imaging Model with PyTorch Lightning
3a. Convert and Quantize a UNet Model and Show Live Inference using POT
3b. Convert and Quantize a UNet Model and Show Live Inference using NNCF
4. Live Inference and Benchmark CT-scan Data with OpenVINO
The main difference between the POT and NNCF quantization notebooks is that NNCF performs quantization within the PyTorch framework, while POT performs quantization after the PyTorch model has been converted to OpenVINO IR format. There is a pre-trained model and a subset of the dataset provided for the quantization notebook, so it is not required to run the data preparation and training notebooks before running the quantization tutorial.
This quantization tutorial consists of the following steps:
- Converting an ONNX model to OpenVINO IR with Model Optimizer.
- Quantizing a model with the Post-Training Optimization Tool API in OpenVINO.
- Evaluating the F1 score metric of the original model and the quantized model.
- Benchmarking performance of the original model and the quantized model.
- Showing live inference with async API and MULTI plugin in OpenVINO.
You will also see real-time segmentation of kidney CT scans running on a CPU, iGPU, or combining both devices for higher throughput. The processed frames are 3D scans that are shown as individual slices. The visualization slides through the slices with detected kidneys overlayed in red. A pre-trained and quantized model is provided, so running the previous notebooks (1-3) in the series is not required.
If you have not installed all required dependencies, follow the Installation Guide.