All categories
11 entries
Quantization
Shrinks models so bigger ones fit in memory.
Where this sits
- QuantizationModelthe weights
- Runtimeruns & routes it
- Harnessmakes it act
- Memorymakes it remember
- Toolsgives it hands
bitsandbytes
4-bit and 8-bit quantization for PyTorch.
8.5K
Core ML Tools
Convert and compress models for Core ML.
Apple Silicon5.4K
TensorRT Model Optimizer
NVIDIA quantization, pruning and distillation.
5.3K
LLM Compressor
Compress models for vLLM serving.
3.9K
Optimum
Hardware-optimized inference and quantization for HF models.
3.5K
torchao
PyTorch-native quantization.
3K
ExLlamaV3
Fast EXL3 quantization and inference on consumer GPUs.
1.7K
AutoRound
Accurate low-bit post-training quantization.
1.6K
AQLM
Extreme compression with additive quantization.
1.3K
GPTQModel
GPTQ quantization toolkit across GPU vendors.
1.3K
Optimum Quanto
Simple PyTorch quantization backend.
1.1K
# Quantization (11) Shrinks models so bigger ones fit in memory. | slug | name | tier | stars | what | md | |---|---|---|---|---|---| | bitsandbytes | bitsandbytes | catalog | 8527 | 4-bit and 8-bit quantization for PyTorch. | /setups/bitsandbytes.md | | core-ml-tools | Core ML Tools | catalog | 5435 | Convert and compress models for Core ML. | /setups/core-ml-tools.md | | tensorrt-model-optimizer | TensorRT Model Optimizer | catalog | 5326 | NVIDIA quantization, pruning and distillation. | /setups/tensorrt-model-optimizer.md | | llm-compressor | LLM Compressor | catalog | 3878 | Compress models for vLLM serving. | /setups/llm-compressor.md | | optimum | Optimum | catalog | 3503 | Hardware-optimized inference and quantization for HF models. | /setups/optimum.md | | torchao | torchao | catalog | 3005 | PyTorch-native quantization. | /setups/torchao.md | | exllamav3 | ExLlamaV3 | catalog | 1673 | Fast EXL3 quantization and inference on consumer GPUs. | /setups/exllamav3.md | | autoround | AutoRound | catalog | 1641 | Accurate low-bit post-training quantization. | /setups/autoround.md | | aqlm | AQLM | catalog | 1330 | Extreme compression with additive quantization. | /setups/aqlm.md | | gptqmodel | GPTQModel | catalog | 1273 | GPTQ quantization toolkit across GPU vendors. | /setups/gptqmodel.md | | optimum-quanto | Optimum Quanto | catalog | 1053 | Simple PyTorch quantization backend. | /setups/optimum-quanto.md |