llama.cpp alternatives.
llama.cpp: The engine behind most local AI. Here are other runtimes & inference tools, most-starred first.
| Tool | What it does | Licence | GitHub stars |
|---|---|---|---|
| Ollama | One command to download and run a model. | MIT | 183,450 |
| vLLM | High-throughput LLM serving engine. | Apache-2.0 | 93,863 |
| SGLang | High-throughput serving for LLMs and VLMs. | Apache-2.0 | 37,175 |
| MLX | Apple's array framework for Apple silicon. | MIT | 28,797 |
| llamafile | Package a model and runtime into one executable. | Other | 26,260 |
| MLC LLM | Compile and run LLMs on any device. | Apache-2.0 | 23,249 |
| KoboldCpp | One-file GGUF runner with a built-in UI. | AGPL-3.0 | 12,009 |
| MLX Examples | Official MLX example models and scripts. | MIT | 9,012 |
| MLX LM | Runs LLMs natively on Apple silicon. | MIT | 7,322 |
| llama-swap | Hot-swaps models behind one OpenAI-style endpoint. | MIT | 5,964 |
| MLX-VLM | Vision-language models on MLX. | MIT | 5,628 |
| LM Studio | A desktop app for local models. | MIT | 5,352 |
Stars are a popularity signal, not a quality score. Gap Scores come after our Mac Studio tests.
# llama.cpp alternatives URL: https://insidethemachine.ai/alternatives/llama-cpp llama.cpp: The engine behind most local AI. Full control over speed, context and quantization. Category: Runtimes & inference. Sorted by GitHub stars (checked 2026-10-16). | name | what | licence | stars | md | |---|---|---|---|---| | Ollama | One command to download and run a model. | MIT | 183450 | https://insidethemachine.ai/setups/ollama.md | | vLLM | High-throughput LLM serving engine. | Apache-2.0 | 93863 | https://insidethemachine.ai/setups/vllm.md | | SGLang | High-throughput serving for LLMs and VLMs. | Apache-2.0 | 37175 | https://insidethemachine.ai/setups/sglang.md | | MLX | Apple's array framework for Apple silicon. | MIT | 28797 | https://insidethemachine.ai/setups/mlx.md | | llamafile | Package a model and runtime into one executable. | Other | 26260 | https://insidethemachine.ai/setups/llamafile.md | | MLC LLM | Compile and run LLMs on any device. | Apache-2.0 | 23249 | https://insidethemachine.ai/setups/mlc-llm.md | | KoboldCpp | One-file GGUF runner with a built-in UI. | AGPL-3.0 | 12009 | https://insidethemachine.ai/setups/koboldcpp.md | | MLX Examples | Official MLX example models and scripts. | MIT | 9012 | https://insidethemachine.ai/setups/mlx-examples.md | | MLX LM | Runs LLMs natively on Apple silicon. | MIT | 7322 | https://insidethemachine.ai/setups/mlx-lm.md | | llama-swap | Hot-swaps models behind one OpenAI-style endpoint. | MIT | 5964 | https://insidethemachine.ai/setups/llama-swap.md | | MLX-VLM | Vision-language models on MLX. | MIT | 5628 | https://insidethemachine.ai/setups/mlx-vlm.md | | LM Studio | A desktop app for local models. | MIT | 5352 | https://insidethemachine.ai/setups/lm-studio.md |