All categories
17 entries
Runtimes & inference
Runs open models on your own hardware.
Where this sits
- Modelthe weights
- RuntimesRuntimeruns & routes it
- Harnessmakes it act
- Memorymakes it remember
- Toolsgives it hands
Runbook
Ollama
One command to download and run a model.
On the channel183K
Runbook
llama.cpp
The engine behind most local AI.
On the channel131K
Runbook
MLX LM
Runs LLMs natively on Apple silicon.
Apple Silicon7.3K
Runbook
LM Studio
A desktop app for local models.
On the channel5.4K
vLLM
High-throughput LLM serving engine.
On the channel94K
SGLang
High-throughput serving for LLMs and VLMs.
37K
MLX
Apple's array framework for Apple silicon.
Apple SiliconOn the channel29K
llamafile
Package a model and runtime into one executable.
26K
MLC LLM
Compile and run LLMs on any device.
23K
KoboldCpp
One-file GGUF runner with a built-in UI.
12K
MLX Examples
Official MLX example models and scripts.
Apple Silicon9K
llama-swap
Hot-swaps models behind one OpenAI-style endpoint.
6K
MLX-VLM
Vision-language models on MLX.
Apple Silicon5.6K
RamaLama
Serve models from any source in containers.
3.1K
vLLM Metal
vLLM plugin for Apple Silicon.
1.9K
MLX OpenAI Server
OpenAI-compatible API server for MLX models.
Apple Silicon362
ML Drift
Google's GPU inference engine for edge devices.
Apple SiliconOn the channel216
# Runtimes & inference (17) Runs open models on your own hardware. | slug | name | tier | stars | what | md | |---|---|---|---|---|---| | ollama | Ollama | runbook | 183450 | One command to download and run a model. | /setups/ollama.md | | llama-cpp | llama.cpp | runbook | 131388 | The engine behind most local AI. | /setups/llama-cpp.md | | mlx-lm | MLX LM | runbook | 7322 | Runs LLMs natively on Apple silicon. | /setups/mlx-lm.md | | lm-studio | LM Studio | runbook | 5352 | A desktop app for local models. | /setups/lm-studio.md | | vllm | vLLM | catalog | 93863 | High-throughput LLM serving engine. | /setups/vllm.md | | sglang | SGLang | catalog | 37175 | High-throughput serving for LLMs and VLMs. | /setups/sglang.md | | mlx | MLX | catalog | 28797 | Apple's array framework for Apple silicon. | /setups/mlx.md | | llamafile | llamafile | catalog | 26260 | Package a model and runtime into one executable. | /setups/llamafile.md | | mlc-llm | MLC LLM | catalog | 23249 | Compile and run LLMs on any device. | /setups/mlc-llm.md | | koboldcpp | KoboldCpp | catalog | 12009 | One-file GGUF runner with a built-in UI. | /setups/koboldcpp.md | | mlx-examples | MLX Examples | catalog | 9012 | Official MLX example models and scripts. | /setups/mlx-examples.md | | llama-swap | llama-swap | catalog | 5964 | Hot-swaps models behind one OpenAI-style endpoint. | /setups/llama-swap.md | | mlx-vlm | MLX-VLM | catalog | 5628 | Vision-language models on MLX. | /setups/mlx-vlm.md | | ramalama | RamaLama | catalog | 3087 | Serve models from any source in containers. | /setups/ramalama.md | | vllm-metal | vLLM Metal | catalog | 1872 | vLLM plugin for Apple Silicon. | /setups/vllm-metal.md | | mlx-openai-server | MLX OpenAI Server | catalog | 362 | OpenAI-compatible API server for MLX models. | /setups/mlx-openai-server.md | | ml-drift | ML Drift | catalog | 216 | Google's GPU inference engine for edge devices. | /setups/ml-drift.md |