The grand list.

# Directory (242 items, 19 categories)

## Runtimes & inference (17)

- ollama: One command to download and run a model. -> /setups/ollama.md
- sglang: High-throughput serving for LLMs and VLMs. -> /setups/sglang.md
- mlx: Apple's array framework for Apple silicon. -> /setups/mlx.md
- llamafile: Package a model and runtime into one executable. -> /setups/llamafile.md
- mlc-llm: Compile and run LLMs on any device. -> /setups/mlc-llm.md
- koboldcpp: One-file GGUF runner with a built-in UI. -> /setups/koboldcpp.md
- mlx-examples: Official MLX example models and scripts. -> /setups/mlx-examples.md
- mlx-lm: Runs LLMs natively on Apple silicon. -> /setups/mlx-lm.md
- llama-swap: Hot-swaps models behind one OpenAI-style endpoint. -> /setups/llama-swap.md
- mlx-vlm: Vision-language models on MLX. -> /setups/mlx-vlm.md
- lm-studio: A desktop app for local models. -> /setups/lm-studio.md
- ramalama: Serve models from any source in containers. -> /setups/ramalama.md
- vllm-metal: vLLM plugin for Apple Silicon. -> /setups/vllm-metal.md
- mlx-openai-server: OpenAI-compatible API server for MLX models. -> /setups/mlx-openai-server.md
- ml-drift: Google's GPU inference engine for edge devices. -> /setups/ml-drift.md
- llama-cpp: The engine behind most local AI. -> /setups/llama-cpp.md
- vllm: High-throughput LLM serving engine. -> /setups/vllm.md

## Model hubs (9)

- transformers: The reference library for open model weights. -> /setups/transformers.md
- unsloth-notebooks: 250+ ready fine-tuning notebooks. -> /setups/unsloth-notebooks.md
- hugging-face-hub: The main hub for open model weights. -> /setups/hugging-face-hub.md
- ollama-library: Ready-to-pull models for Ollama. -> /setups/ollama-library.md
- lm-studio-models: Curated models you can load in LM Studio. -> /setups/lm-studio-models.md
- mlx-community: Community MLX conversions of popular models. -> /setups/mlx-community.md
- unsloth-models: Unsloth GGUF and fine-tune-ready model uploads. -> /setups/unsloth-models.md
- lmstudio-community: LM Studio's community GGUF/MLX uploads. -> /setups/lmstudio-community.md
- kaggle-models: Google-run model hub with downloadable weights. -> /setups/kaggle-models.md

## Models from the channel (6)

- qwen-image-2-1-turbo: 7B image model that paints and edits in 8 steps. -> /setups/qwen-image-2-1-turbo.md
- mellum-2-1: JetBrains coding model, 12B with 2.5B active. -> /setups/mellum-2-1.md
- embeddinggemma-2: 740M on-device embedding model for search. -> /setups/embeddinggemma-2.md
- kolibri-1: Open German/English MoE model (about 78 GB of memory, per the video). -> /setups/kolibri-1.md
- astabrief-8b: Small Ai2 model that writes cited reports. -> /setups/astabrief-8b.md
- cloudflare-clef: Cloudflare's small open decision models (Clef, Clef-flash). -> /setups/cloudflare-clef.md

## Coding agents (16)

- opencode: Open-source coding agent for your terminal. -> /setups/opencode.md
- pi-pi-mono: Small agent toolkit: LLM API, agent loop and coding CLI. -> /setups/pi-pi-mono.md
- goose: Extensible open-source agent that runs code and tools. -> /setups/goose.md
- aider: A coding agent in your terminal. -> /setups/aider.md
- mini-swe-agent: A 100-line agent that fixes GitHub issues. -> /setups/mini-swe-agent.md
- claude-code: Anthropic's terminal coding agent (cloud model). -> /setups/claude-code.md
- gemini-cli: Google's open-source terminal agent for Gemini. -> /setups/gemini-cli.md
- openhands: Open platform for autonomous software agents. -> /setups/openhands.md
- cline: Coding agent as a VS Code extension, CLI or SDK. -> /setups/cline.md
- continue: Open-source coding agent for IDEs and CI. -> /setups/continue.md
- tabby: Self-hosted code completion and chat server. -> /setups/tabby.md
- crush: Polished terminal coding agent from Charm. -> /setups/crush.md
- qwen-code: Qwen's open-source terminal coding agent. -> /setups/qwen-code.md
- kilo-code: All-in-one agentic coding extension and CLI. -> /setups/kilo-code.md
- codex-cli: OpenAI's lightweight terminal coding agent. -> /setups/codex-cli.md
- copilot-cli: GitHub's terminal coding agent; can use Ollama models. -> /setups/copilot-cli.md

## General & personal agents (15)

- openclaw: Personal agent that runs on your machine and chats anywhere. -> /setups/openclaw.md
- hermes-agent: A personal agent that learns skills and remembers you. -> /setups/hermes-agent.md
- autogpt: Platform for building autonomous agents. -> /setups/autogpt.md
- browser-use: Let an AI agent drive a web browser. -> /setups/browser-use.md
- deerflow: Long-running research and coding super-agent. -> /setups/deerflow.md
- open-interpreter: Agent that runs code on your computer, for open models. -> /setups/open-interpreter.md
- openmanus: Open-source general agent (Manus-style). -> /setups/openmanus.md
- nanobot: Ultra-light self-hosted personal agent. -> /setups/nanobot.md
- ui-tars-desktop: Multimodal agent that operates your desktop. -> /setups/ui-tars-desktop.md
- khoj: Self-hostable AI second brain over your docs and the web. -> /setups/khoj.md
- perplexica: Open answering engine (formerly Perplexica). -> /setups/perplexica.md
- nanoclaw: Lightweight, container-sandboxed OpenClaw alternative. -> /setups/nanoclaw.md
- gpt-researcher: Agent that writes deep research reports. -> /setups/gpt-researcher.md
- cua: Open-source computer-use drivers and sandboxes. -> /setups/cua.md
- lm-studio-bionic: LM Studio's agent that can run beside the app. -> /setups/lm-studio-bionic.md

## Agent frameworks (14)

- litellm: One API in front of every model, local or cloud. -> /setups/litellm.md
- autogen: Microsoft's multi-agent programming framework. -> /setups/autogen.md
- crewai: Role-based multi-agent framework. -> /setups/crewai.md
- langgraph: Build stateful, resilient agent graphs. -> /setups/langgraph.md
- agno: Build and run agent systems in Python. -> /setups/agno.md
- dspy: Program LLMs instead of hand-writing prompts. -> /setups/dspy.md
- openai-agents-sdk: OpenAI's lightweight multi-agent SDK. -> /setups/openai-agents-sdk.md
- smolagents: Tiny library for agents that think in code. -> /setups/smolagents.md
- mastra: TypeScript framework for agents and workflows. -> /setups/mastra.md
- vercel-ai-sdk: TypeScript toolkit for AI apps and agents. -> /setups/vercel-ai-sdk.md
- haystack: Orchestration framework for RAG and agents. -> /setups/haystack.md
- letta: Stateful agents with long-term memory (ex-MemGPT). -> /setups/letta.md
- google-adk: Google's code-first agent development kit. -> /setups/google-adk.md
- pydantic-ai: Typed Python agent framework. -> /setups/pydantic-ai.md

## Memory (11)

- mem0: Long-term memory for your agent. -> /setups/mem0.md
- claude-mem: Captures agent sessions and replays context later. -> /setups/claude-mem.md
- cognee: Knowledge-graph memory for agents. -> /setups/cognee.md
- graphiti: Real-time temporal knowledge graphs for agents. -> /setups/graphiti.md
- supermemory: Fast memory engine and app, can run locally. -> /setups/supermemory.md
- memos: Memory operating system for LLM agents. -> /setups/memos.md
- honcho: Memory library that models users across sessions. -> /setups/honcho.md
- basic-memory: Markdown-file memory your agent reads and writes. -> /setups/basic-memory.md
- memobase: User-profile memory for chat apps. -> /setups/memobase.md
- memsearch: Semantic search over agent memory files. -> /setups/memsearch.md
- mcp-memory-service: Persistent memory exposed as an MCP server. -> /setups/mcp-memory-service.md

## RAG & document parsing (14)

- docling: Turns documents into clean text for AI. -> /setups/docling.md
- marker: Fast, accurate PDF-to-markdown converter. -> /setups/marker.md
- lightrag: Simple, fast graph-based RAG. -> /setups/lightrag.md
- qdrant: High-performance vector database. -> /setups/qdrant.md
- chroma: Open-source embedding database. -> /setups/chroma.md
- pgvector: Vector search inside Postgres. -> /setups/pgvector.md
- lancedb: Embedded multimodal vector database. -> /setups/lancedb.md
- sqlite-vec: Vector search as a tiny SQLite extension. -> /setups/sqlite-vec.md
- markitdown: Convert files and Office docs to markdown. -> /setups/markitdown.md
- ragflow: Deep-document RAG engine with a UI. -> /setups/ragflow.md
- paddleocr: OCR and document parsing toolkit. -> /setups/paddleocr.md
- mineru: Converts complex PDFs to markdown or JSON. -> /setups/mineru.md
- anythingllm: All-in-one desktop app for chat with your docs. -> /setups/anythingllm.md
- meilisearch: Fast search engine with hybrid vector search. -> /setups/meilisearch.md

## Routing & gateways (10)

- strands-decider: Small decision model that routes agent choices. -> /setups/strands-decider.md
- new-api: Unified gateway that converts many LLM APIs. -> /setups/new-api.md
- one-api: LLM API management and distribution gateway. -> /setups/one-api.md
- portkey-gateway: Fast AI gateway with guardrails and fallbacks. -> /setups/portkey-gateway.md
- bifrost: High-performance LLM and MCP gateway. -> /setups/bifrost.md
- plano: AI-native proxy with smart LLM routing. -> /setups/plano.md
- llmrouter: Research library of LLM routing methods. -> /setups/llmrouter.md
- envoy-ai-gateway: Envoy-based gateway for generative AI services. -> /setups/envoy-ai-gateway.md
- nvidia-llm-router: Routes each prompt to the best-fit model. -> /setups/nvidia-llm-router.md
- nvidia-pair: Shares AI requests across the computers on your home network. -> /setups/nvidia-pair.md

## MCP servers & tools (14)

- mcp-servers: Standard plug-in tools for agents. -> /setups/mcp-servers.md
- mcp-python-sdk: Official Python SDK for MCP servers and clients. -> /setups/mcp-python-sdk.md
- llm-cli: CLI and Python library for any LLM, local or cloud. -> /setups/llm-cli.md
- mcp-inspector: Visual tester for MCP servers. -> /setups/mcp-inspector.md
- n8n: Workflow automation with AI nodes, self-hostable. -> /setups/n8n.md
- firecrawl: Turn websites into LLM-ready data. -> /setups/firecrawl.md
- awesome-mcp-servers: Big curated list of MCP servers. -> /setups/awesome-mcp-servers.md
- crawl4ai: Open-source crawler that outputs clean markdown. -> /setups/crawl4ai.md
- context7: Up-to-date library docs for coding agents. -> /setups/context7.md
- chrome-devtools-mcp: Chrome DevTools as an MCP server. -> /setups/chrome-devtools-mcp.md
- searxng: Self-hosted private metasearch engine. -> /setups/searxng.md
- playwright-mcp: Browser automation MCP server. -> /setups/playwright-mcp.md
- github-mcp-server: GitHub's official MCP server. -> /setups/github-mcp-server.md
- serena: Semantic code retrieval and editing over MCP. -> /setups/serena.md

## Chat UIs & front-ends (14)

- open-webui: A ChatGPT-style app for your local models. -> /setups/open-webui.md
- nextchat: Lightweight cross-platform chat client. -> /setups/nextchat.md
- lobechat: Agent workspace and chat UI. -> /setups/lobechat.md
- cherry-studio: Desktop AI studio for many providers. -> /setups/cherry-studio.md
- text-generation-webui: Desktop app for local LLMs (text-generation-webui). -> /setups/text-generation-webui.md
- librechat: Self-hosted multi-provider ChatGPT clone. -> /setups/librechat.md
- jan: Offline ChatGPT alternative for your desktop. -> /setups/jan.md
- chatbox: Desktop and mobile AI client. -> /setups/chatbox.md
- sillytavern: Power-user chat and roleplay front-end. -> /setups/sillytavern.md
- open-llm-vtuber: Voice chat with an animated avatar, locally. -> /setups/open-llm-vtuber.md
- hf-chat-ui: The open code behind HuggingChat. -> /setups/hf-chat-ui.md
- big-agi: Advanced multi-model chat suite. -> /setups/big-agi.md
- enchanted: Native macOS/iOS app for self-hosted models. -> /setups/enchanted.md
- llama-vscode: Local code completion in VS Code via llama.cpp. -> /setups/llama-vscode.md

## Voice: speech-to-text & TTS (14)

- whisper-cpp: Speech-to-text that runs on your machine. -> /setups/whisper-cpp.md
- mlx-audio: TTS, STT and speech-to-speech on MLX. -> /setups/mlx-audio.md
- whisperkit: On-device speech AI for Apple Silicon (WhisperKit). -> /setups/whisperkit.md
- parakeet-mlx: NVIDIA Parakeet speech recognition on MLX. -> /setups/parakeet-mlx.md
- whisper: OpenAI's original speech recognition model. -> /setups/whisper.md
- chattts: Conversational text-to-speech model. -> /setups/chattts.md
- handy: Offline push-to-talk dictation app. -> /setups/handy.md
- fish-speech: Open-source text-to-speech with voice cloning. -> /setups/fish-speech.md
- chatterbox: Open-source expressive TTS. -> /setups/chatterbox.md
- faster-whisper: Whisper transcription, faster, via CTranslate2. -> /setups/faster-whisper.md
- whisperx: Whisper with word timestamps and speaker labels. -> /setups/whisperx.md
- cosyvoice: Multilingual voice generation model. -> /setups/cosyvoice.md
- dia: TTS that generates realistic dialogue. -> /setups/dia.md
- nemo-parakeet: NVIDIA NeMo speech models (incl. Parakeet). -> /setups/nemo-parakeet.md

## Image & video generation (14)

- stable-diffusion-cpp: Image and video diffusion in pure C/C++. -> /setups/stable-diffusion-cpp.md
- mflux: MLX-native image and video models for Mac. -> /setups/mflux.md
- qwen-image-2-1: Qwen's open image generation and editing model. -> /setups/qwen-image-2-1.md
- draw-things: Community repo for the Draw Things Mac/iOS app. -> /setups/draw-things.md
- stable-diffusion-webui: The classic Stable Diffusion web UI. -> /setups/stable-diffusion-webui.md
- comfyui: Node-based GUI and backend for diffusion models. -> /setups/comfyui.md
- fooocus: Prompt-first, low-setup image generator. -> /setups/fooocus.md
- diffusers: PyTorch library for image, video and audio diffusion. -> /setups/diffusers.md
- open-sora: Open video generation models. -> /setups/open-sora.md
- invokeai: Professional creative studio for diffusion. -> /setups/invokeai.md
- core-ml-stable-diffusion: Stable Diffusion on Core ML for Apple Silicon. -> /setups/core-ml-stable-diffusion.md
- wan-2-2: Open video generation models (Wan 2.2). -> /setups/wan-2-2.md
- wan-2-1: Open video generation models (Wan 2.1). -> /setups/wan-2-1.md
- comfyui-manager: Install and manage ComfyUI custom nodes. -> /setups/comfyui-manager.md

## Fine-tuning (14)

- mlx-lm-lora: LoRA and full fine-tuning on MLX. -> /setups/mlx-lm-lora.md
- unsloth: Fast, low-memory fine-tuning with a local UI. -> /setups/unsloth.md
- llama-factory: Fine-tune 100+ LLMs and VLMs from one tool. -> /setups/llama-factory.md
- nanogpt: The simplest repo to train small GPTs. -> /setups/nanogpt.md
- nanochat: Train a small ChatGPT end to end on a budget. -> /setups/nanochat.md
- open-r1: Open reproduction of DeepSeek-R1 training. -> /setups/open-r1.md
- verl: Reinforcement-learning post-training framework. -> /setups/verl.md
- peft: LoRA and other parameter-efficient fine-tuning. -> /setups/peft.md
- trl: Train models with SFT, DPO and RL. -> /setups/trl.md
- ms-swift: Fine-tune 600+ LLMs with PEFT or full training. -> /setups/ms-swift.md
- axolotl: Config-driven fine-tuning toolkit. -> /setups/axolotl.md
- openrlhf: Scalable RLHF and agentic RL on Ray. -> /setups/openrlhf.md
- oumi: Fine-tune, evaluate and deploy open models. -> /setups/oumi.md
- mergekit: Merge model weights into new models. -> /setups/mergekit.md

## Quantization (11)

- bitsandbytes: 4-bit and 8-bit quantization for PyTorch. -> /setups/bitsandbytes.md
- core-ml-tools: Convert and compress models for Core ML. -> /setups/core-ml-tools.md
- tensorrt-model-optimizer: NVIDIA quantization, pruning and distillation. -> /setups/tensorrt-model-optimizer.md
- llm-compressor: Compress models for vLLM serving. -> /setups/llm-compressor.md
- optimum: Hardware-optimized inference and quantization for HF models. -> /setups/optimum.md
- torchao: PyTorch-native quantization. -> /setups/torchao.md
- exllamav3: Fast EXL3 quantization and inference on consumer GPUs. -> /setups/exllamav3.md
- autoround: Accurate low-bit post-training quantization. -> /setups/autoround.md
- aqlm: Extreme compression with additive quantization. -> /setups/aqlm.md
- gptqmodel: GPTQ quantization toolkit across GPU vendors. -> /setups/gptqmodel.md
- optimum-quanto: Simple PyTorch quantization backend. -> /setups/optimum-quanto.md

## Evals & benchmarks (14)

- thinkingbox: Runs agents against MCP tools and checks what they really did. -> /setups/thinkingbox.md
- promptfoo: Test and red-team prompts, agents and RAG. -> /setups/promptfoo.md
- openai-evals: Framework and registry for evaluating LLMs. -> /setups/openai-evals.md
- deepeval: Unit-test style evaluation for LLM apps. -> /setups/deepeval.md
- ragas: Metrics for evaluating RAG pipelines. -> /setups/ragas.md
- lm-evaluation-harness: The standard harness for academic LLM benchmarks. -> /setups/lm-evaluation-harness.md
- gorilla-bfcl: Function-calling models and the Berkeley tool-use leaderboard. -> /setups/gorilla-bfcl.md
- opencompass: Large evaluation platform across many benchmarks. -> /setups/opencompass.md
- swe-bench: Benchmark of real GitHub issues for coding agents. -> /setups/swe-bench.md
- simple-evals: OpenAI's lightweight reference evals. -> /setups/simple-evals.md
- lmms-eval: Evaluation toolkit for multimodal models. -> /setups/lmms-eval.md
- inspect-ai: UK AISI framework for LLM and agent evals. -> /setups/inspect-ai.md
- terminal-bench: Benchmark of hard tasks in a real terminal. -> /setups/terminal-bench.md
- lighteval: Hugging Face's multi-backend eval toolkit. -> /setups/lighteval.md

## Scaffolds & templates (14)

- agents-md: A README written for AI agents. -> /setups/agents-md.md
- clawhub: Skill and plugin registry for OpenClaw. -> /setups/clawhub.md
- superpowers: Agent skills library and dev methodology. -> /setups/superpowers.md
- anthropic-skills: Anthropic's public Agent Skills examples. -> /setups/anthropic-skills.md
- spec-kit: Spec-driven development toolkit for agents. -> /setups/spec-kit.md
- openspec: Lightweight spec-driven workflow for coding agents. -> /setups/openspec.md
- awesome-claude-code: Curated Claude Code commands, hooks and workflows. -> /setups/awesome-claude-code.md
- bmad-method: Agile multi-agent development method. -> /setups/bmad-method.md
- awesome-cursor-rules: Collection of agent rule files. -> /setups/awesome-cursor-rules.md
- wshobson-agents: Multi-harness agent plugin marketplace. -> /setups/wshobson-agents.md
- claude-code-templates: CLI to install agent configs and templates. -> /setups/claude-code-templates.md
- vercel-agent-skills: Vercel's official agent skills. -> /setups/vercel-agent-skills.md
- task-master: Task-management system for coding agents. -> /setups/task-master.md
- soul-md: A file that sets your agent's identity and voice. -> /setups/soul-md.md

## Observability (12)

- langfuse: Open-source tracing and evals for LLM apps. -> /setups/langfuse.md
- mlflow: ML and LLM lifecycle platform with tracing. -> /setups/mlflow.md
- opik: Trace, evaluate and monitor LLM apps. -> /setups/opik.md
- phoenix: Open-source LLM tracing and evaluation. -> /setups/phoenix.md
- evidently: Evaluate and monitor ML and LLM systems. -> /setups/evidently.md
- openllmetry: OpenTelemetry instrumentation for LLM apps. -> /setups/openllmetry.md
- helicone: LLM observability via a one-line proxy. -> /setups/helicone.md
- agentops: Monitoring and cost tracking for agents. -> /setups/agentops.md
- agenta: Prompt management, evals and observability. -> /setups/agenta.md
- logfire: Observability for LLM and agent systems. -> /setups/logfire.md
- openlit: OpenTelemetry-native tracing and guardrails. -> /setups/openlit.md
- weave: W&B toolkit for tracing and evaluating AI apps. -> /setups/weave.md

## Hardware & system utilities (9)

- apple-container: Run Linux containers natively on a Mac. -> /setups/apple-container.md
- stats: macOS menu-bar system monitor. -> /setups/stats.md
- llmfit: One command to see which models fit your hardware. -> /setups/llmfit.md
- nvtop: GPU monitor for Apple, AMD, Intel and NVIDIA. -> /setups/nvtop.md
- nvitop: Interactive NVIDIA GPU process viewer. -> /setups/nvitop.md
- gpustack: GPU cluster manager for model serving. -> /setups/gpustack.md
- nvidia-container-toolkit: Use NVIDIA GPUs inside containers. -> /setups/nvidia-container-toolkit.md
- mactop: Apple Silicon CPU/GPU/power monitor in the terminal. -> /setups/mactop.md
- gguf-parser: Inspect GGUF files and estimate memory use. -> /setups/gguf-parser.md