The most comprehensive GUI tool for GGUF model quantization. Stop wrestling with command lines - quantize, merge, and optimize your models with just a few clicks.
- 📩 Download and manage llama.cpp backends
- 🗃️ Select and quantize GGUF models
- 📐 Configure quantization parameters
- 💻 Monitor system resources during quantization
- ⏳ Parallel quantization + imatrix generation
- 🎉 LoRA conversion and merging
- 📁 Preset saving and loading
- 8️⃣ AutoFP8 quantization
- 🪓 GGUF splitting and merging
- 🌐 HTTP API for automated monitoring
- Fast: Saves time on manual configuration
- Simple: Clean UI, no terminal needed
- Powerful: Handles models up to infinite size, only limited by your RAM
- Resource-aware: Optimized memory management and efficient UI library
git clone https://github.com/leafspark/AutoGGUFcd AutoGGUF- Install dependencies:
pip install -r requirements.txt - Run the application:
or use the
python src/main.pyrun.batscript.
macOS and Ubuntu builds are provided with GitHub Actions, you may download the binaries in the releases section.
Standard builds:
- Download the latest release
- Extract all files to a folder
- Run
AutoGGUF-x64.exe - Any necessary folders will be automatically created
Setup builds:
- Download setup variant of latest release
- Extract all files to a folder
- Run the setup program
- The .GGUF extension will be registered with the program automatically
- Run the program from the Start Menu or desktop shortcuts
After launching the program, you may access its local server at port 7001 (set AUTOGGUF_SERVER to "enabled" first)
gpg --import AutoGGUF-v1.5.0-prerel.asc
gpg --verify AutoGGUF-v1.9.1-Windows-avx2.zip.sig AutoGGUF-v1.9.1-Windows-avx2.zip
sha256sum -c AutoGGUF-v1.9.1.sha256# Import the public key
gpg --import AutoGGUF-v1.5.0-prerel.asc
# Verify the signature
gpg --verify AutoGGUF-v1.9.1-Windows-avx2.zip.sig AutoGGUF-v1.9.1-Windows-avx2.zip
# Check SHA256
$fileHash = (Get-FileHash -Algorithm SHA256 AutoGGUF-v1.9.1-Windows-avx2.zip).Hash.ToLower()
$storedHash = (Get-Content AutoGGUF-v1.9.1.sha256 | Select-String AutoGGUF-v1.9.1-Windows-avx2.zip).Line.Split()[0]
if ($fileHash -eq $storedHash) { "SHA256 Match" } else { "SHA256 Mismatch" }Release keys are identical to ones used for commiting.
pip install -U pyinstaller
./build.sh RELEASE | DEV
cd build/<type>/dist/
./AutoGGUFpip install -U pyinstaller
build RELEASE | DEVFind the executable in build/<type>/dist/AutoGGUF-x64.exe.
You can also use Nuitka, which may result in a slower build but a faster output executable:
build_optimized RELEASE | DEVView the list of supported languages at AutoGGUF/wiki/Installation#configuration (LLM translated, except for English).
More languages will be updated as soon as possible!
To use a specific language, set the AUTOGGUF_LANGUAGE environment variable to one of the listed language codes (note: some languages may not be fully supported yet, those will fall back to English).
- Some inconsistent logging
- Missing translations
- Time estimation for quantization
- Quantization file size estimate
- Perplexity testing
- bitsandbytes
Due to my limited availability and a lack of time, I won't be actively developing new features for this project as much. While I'll continue to publish builds from time to time, I strongly recommend running from source if you want to stay up to date with the latest changes. I'm still committed to keeping dependencies updated weekly and making small maintenance fixes to ensure everything runs smoothly. If you run into any problems or notice issues, please don't hesitate to let me know - I appreciate your feedback and will do my best to address them.
- SSL module cannot be found error: Install OpenSSL or run from source using
python src/main.pywith therun.batscript (pip install requests) - Check out the Wiki for advanced usage and configuration
Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description. Read CONTRIBUTING.md for more information.