Running ONNX models

[ndrean]

Misc readings.

Not all models are available in Hugging Face. You may use AxonONNX or ORTEX to run ONNX models models.

1. From PyTorch to Axon

This video is interesting because it explains how to use a model that is not necessarily directly usable into Bumblebee (eg models from Hugging Face).
It takes a PyTorch based pre-trained model (meaning the architecture of layers and operations with their parameters) and serialises into the ONNX format. This model is then imported into the Elixir world with axon_onnx.

Note that you need to know what the model expects.

They used Kino.Input in the Livebook to read and get an Nx.tensor from an image, but you would use StbImage or, for more advanced features, Vix or Image.

2. Livebook from Axon docs

This livebook uses only Axon and StbImage to load an ONNX model and run predictions.

3. Blog on Ortex

You may skip the steps above.
The blog post below explains that you can use the ONNX runtime bindings to load and run directly an ONNX model and get an Nx.tensor.