# Inside The Machine

> Agent-first directory of 242 local-AI tools, models and repos (runtimes, agents, memory, RAG, routing, MCP, UIs, voice, image/video, fine-tuning, quantization, evals, scaffolds, observability, hardware utilities). 13 "runbook" listings have install/verify/rollback checked against official docs; the rest are "catalog" entries with verified metadata and unverified, README-extracted install commands. Gap Scores are pending test on Mac Studio.

Append .md or .json to any /setups/<slug> URL, or send Accept: text/markdown.

## Start here

- [Zero to working local AI](https://insidethemachine.ai/start.md): 5-step runbook + AGENTS.md/SOUL.md + lean-context guide. JSON: https://insidethemachine.ai/start.json
- [Models vs memory](https://insidethemachine.ai/models.md): which open models fit which RAM tier. JSON: https://insidethemachine.ai/models.json (124 text models; plus speech.json and images.json)
- [Machines](https://insidethemachine.ai/machines.md): hardware guide. JSON: https://insidethemachine.ai/machines.json

## Best picks (answers with dated data)

- [Best local LLM by memory](https://insidethemachine.ai/best.md): one table, 8 to 512 GB: models that fit, best model, best coding model
- [Best local LLM for 8 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-8gb-ram.md): top picks, coding/agent/vision pick, machines with 8 GB
- [Best local LLM for 16 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-16gb-ram.md): top picks, coding/agent/vision pick, machines with 16 GB
- [Best local LLM for 24 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-24gb-ram.md): top picks, coding/agent/vision pick, machines with 24 GB
- [Best local LLM for 32 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-32gb-ram.md): top picks, coding/agent/vision pick, machines with 32 GB
- [Best local LLM for 48 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-48gb-ram.md): top picks, coding/agent/vision pick, machines with 48 GB
- [Best local LLM for 64 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-64gb-ram.md): top picks, coding/agent/vision pick, machines with 64 GB
- [Best local LLM for 96 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-96gb-ram.md): top picks, coding/agent/vision pick, machines with 96 GB
- [Best local LLM for 128 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-128gb-ram.md): top picks, coding/agent/vision pick, machines with 128 GB
- [Best local LLM for 192 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-192gb-ram.md): top picks, coding/agent/vision pick, machines with 192 GB
- [Best local LLM for 256 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-256gb-ram.md): top picks, coding/agent/vision pick, machines with 256 GB
- [Best local LLM for 512 GB RAM or VRAM](https://insidethemachine.ai/best/local-llm-for-512gb-ram.md): top picks, coding/agent/vision pick, machines with 512 GB
- [Best local coding model](https://insidethemachine.ai/best/local-coding-model.md): per memory size, Terminal-Bench, coding apps
- [Best local model for agents](https://insidethemachine.ai/best/local-model-for-agents.md): OpenClaw, Hermes Agent and more
- [Best local LLM for Mac](https://insidethemachine.ai/best/local-llm-for-mac.md) · [NVIDIA GPU](https://insidethemachine.ai/best/local-llm-for-nvidia-gpu.md) · [mini PC](https://insidethemachine.ai/best/local-llm-for-mini-pc.md)
- [Compare apps](https://insidethemachine.ai/compare.md): [Ollama vs LM Studio](https://insidethemachine.ai/compare/ollama-vs-lm-studio.md) · [Ollama vs llama.cpp](https://insidethemachine.ai/compare/ollama-vs-llama-cpp.md) · [LM Studio vs llama.cpp](https://insidethemachine.ai/compare/lm-studio-vs-llama-cpp.md) · [MLX LM vs Ollama](https://insidethemachine.ai/compare/mlx-lm-vs-ollama.md) · [Open WebUI vs LM Studio](https://insidethemachine.ai/compare/open-webui-vs-lm-studio.md)
- Alternatives: [Ollama](https://insidethemachine.ai/alternatives/ollama.md) · [MLX LM](https://insidethemachine.ai/alternatives/mlx-lm.md) · [LM Studio](https://insidethemachine.ai/alternatives/lm-studio.md) · [llama.cpp](https://insidethemachine.ai/alternatives/llama-cpp.md) · [Aider](https://insidethemachine.ai/alternatives/aider.md) · [Hermes Agent](https://insidethemachine.ai/alternatives/hermes-agent.md) · [LiteLLM](https://insidethemachine.ai/alternatives/litellm.md) · [Mem0](https://insidethemachine.ai/alternatives/mem0.md) · [Docling](https://insidethemachine.ai/alternatives/docling.md) · [Open WebUI](https://insidethemachine.ai/alternatives/open-webui.md) · [whisper.cpp](https://insidethemachine.ai/alternatives/whisper-cpp.md)
- Machines (one page each): [MacBook Air 13"/15" (M5)](https://insidethemachine.ai/machines/macbook-air-m5.md) · [MacBook Pro 14" (M5)](https://insidethemachine.ai/machines/macbook-pro-14-m5.md) · [MacBook Pro 14"/16" (M5 Pro)](https://insidethemachine.ai/machines/macbook-pro-m5-pro.md) · [MacBook Pro 14"/16" (M5 Max)](https://insidethemachine.ai/machines/macbook-pro-m5-max.md) · [Mac mini (M6)](https://insidethemachine.ai/machines/mac-mini-m6.md) · [Mac mini (M5 Pro)](https://insidethemachine.ai/machines/mac-mini-m5-pro.md) · [Mac Studio (M5 Max)](https://insidethemachine.ai/machines/mac-studio-m5-max.md) · [Mac Studio (M5 Ultra)](https://insidethemachine.ai/machines/mac-studio-m5-ultra.md) · [GeForce RTX 5090](https://insidethemachine.ai/machines/rtx-5090.md) · [GeForce RTX 5080](https://insidethemachine.ai/machines/rtx-5080.md) · [GeForce RTX 5070 Ti](https://insidethemachine.ai/machines/rtx-5070-ti.md) · [GeForce RTX 5070](https://insidethemachine.ai/machines/rtx-5070.md) · [GeForce RTX 5060 Ti 16GB](https://insidethemachine.ai/machines/rtx-5060-ti-16gb.md) · [GeForce RTX 5060](https://insidethemachine.ai/machines/rtx-5060.md) · [GeForce RTX 4090](https://insidethemachine.ai/machines/rtx-4090.md) · [GeForce RTX 4080 Super](https://insidethemachine.ai/machines/rtx-4080-super.md) · [GeForce RTX 4070 Ti Super](https://insidethemachine.ai/machines/rtx-4070-ti-super.md) · [GeForce RTX 4060 Ti 16GB](https://insidethemachine.ai/machines/rtx-4060-ti-16gb.md) · [GeForce RTX 3090 (used)](https://insidethemachine.ai/machines/rtx-3090.md) · [GeForce RTX 3090 Ti](https://insidethemachine.ai/machines/rtx-3090-ti.md) · [GeForce RTX 3060 12GB (used)](https://insidethemachine.ai/machines/rtx-3060-12gb.md) · [RTX PRO 6000 Blackwell Workstation (96GB)](https://insidethemachine.ai/machines/rtx-pro-6000-blackwell.md) · [RTX PRO 6000 Blackwell Max-Q (96GB)](https://insidethemachine.ai/machines/rtx-pro-6000-blackwell-maxq.md) · [RTX PRO 5000 Blackwell 72GB](https://insidethemachine.ai/machines/rtx-pro-5000-blackwell-72gb.md) · [RTX PRO 4500 Blackwell 32GB](https://insidethemachine.ai/machines/rtx-pro-4500-blackwell.md) · [RTX PRO 4000 Blackwell 24GB](https://insidethemachine.ai/machines/rtx-pro-4000-blackwell.md) · [NVIDIA DGX Spark (128GB)](https://insidethemachine.ai/machines/dgx-spark-128gb.md) · [DGX Spark 64GB (partner GB10 systems)](https://insidethemachine.ai/machines/dgx-spark-64gb.md) · [NVIDIA Jetson AGX Thor Developer Kit](https://insidethemachine.ai/machines/jetson-agx-thor.md) · [NVIDIA Jetson Orin Nano Super Developer Kit](https://insidethemachine.ai/machines/jetson-orin-nano-super.md) · [Framework Desktop (Ryzen AI Max+ 395)](https://insidethemachine.ai/machines/framework-desktop-395.md) · [GMKtec EVO-X2 (Ryzen AI Max+ 395)](https://insidethemachine.ai/machines/gmktec-evo-x2.md) · [Minisforum MS-S1 MAX (Ryzen AI Max+ 395)](https://insidethemachine.ai/machines/minisforum-ms-s1-max.md) · [Bosgame M5 (Ryzen AI Max+ 395)](https://insidethemachine.ai/machines/bosgame-m5.md) · [Beelink GTR9 Pro (Ryzen AI Max+ 395)](https://insidethemachine.ai/machines/beelink-gtr9-pro.md) · [GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495 "Gorgon Halo")](https://insidethemachine.ai/machines/gmktec-evo-x5-pro.md) · [Radeon RX 7900 XTX](https://insidethemachine.ai/machines/radeon-rx-7900-xtx.md) · [Radeon RX 9070 XT](https://insidethemachine.ai/machines/radeon-rx-9070-xt.md) · [Radeon AI PRO R9700 (32GB)](https://insidethemachine.ai/machines/radeon-ai-pro-r9700.md) · [Intel Arc B580](https://insidethemachine.ai/machines/intel-arc-b580.md) · [Intel Arc Pro B60 (24GB)](https://insidethemachine.ai/machines/intel-arc-pro-b60.md) · [Intel Arc Pro B70 (32GB)](https://insidethemachine.ai/machines/intel-arc-pro-b70.md) · [Raspberry Pi 5 (16GB), CPU only](https://insidethemachine.ai/machines/raspberry-pi-5-16gb.md) · [Tesla P40 24GB (used)](https://insidethemachine.ai/machines/tesla-p40.md) · [AMD Instinct MI50 32GB (used)](https://insidethemachine.ai/machines/instinct-mi50-32gb.md) · [Tesla V100 PCIe 32GB (used)](https://insidethemachine.ai/machines/tesla-v100-32gb.md) · [Minisforum MS-S1 MAX-P495 (Ryzen AI Max+ PRO 495 "Gorgon Halo")](https://insidethemachine.ai/machines/minisforum-ms-s1-max-p495.md) · [Gigabyte W775-V10-L01 (DGX Station GB300)](https://insidethemachine.ai/machines/gigabyte-w775-v10-l01.md) · [ASUS ProArt GR1X (NVIDIA RTX Spark)](https://insidethemachine.ai/machines/asus-proart-gr1x.md) · [iBUYPOWER Pro AMD AI Halo Mini PC (Ryzen AI Max+ PRO 495)](https://insidethemachine.ai/machines/ibuypower-pro-amd-ai-halo.md) · [Microsoft Surface RTX Spark Dev Box (128GB)](https://insidethemachine.ai/machines/surface-rtx-spark-dev-box.md) · [AMD Ryzen AI Halo Developer Platform (Ryzen AI Max+ PRO 495, 192GB)](https://insidethemachine.ai/machines/amd-ryzen-ai-halo-495.md) · [Sipeed MaixStation](https://insidethemachine.ai/machines/sipeed-maixstation.md)

## Your machine, one-command installs, what changed

- [Scan my machine](https://insidethemachine.ai/scan.sh): read-only; prints a https://insidethemachine.ai/my?... link. Windows: https://insidethemachine.ai/scan.ps1. Report for agents: https://insidethemachine.ai/my.json?ram=<GB>&os=<mac|linux|win>&chip=<slug>&gpu=<slug>&vram=<GB> (also /my.md)
- [One-command installs](https://insidethemachine.ai/install.md): 19 scripts with install, verify, rollback and an honest run status. JSON: https://insidethemachine.ai/install.json
- [Install: A private ChatGPT](https://insidethemachine.ai/install/chat.md) [ran]: `curl -fsSL https://insidethemachine.ai/install/chat.sh | bash`
- [Install: A coding helper](https://insidethemachine.ai/install/code.md) [partly-ran]: `curl -fsSL https://insidethemachine.ai/install/code.sh | bash`
- [Install: Chat with your files](https://insidethemachine.ai/install/docs.md) [partly-ran]: `curl -fsSL https://insidethemachine.ai/install/docs.sh | bash`
- [Install: Speech to text](https://insidethemachine.ai/install/voice.md) [ran]: `curl -fsSL https://insidethemachine.ai/install/voice.sh | bash`
- [Install: Make images](https://insidethemachine.ai/install/images.md) [not-run]: `curl -fsSL https://insidethemachine.ai/install/images.sh | bash`
- [Install: An agent that does tasks](https://insidethemachine.ai/install/agent.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/agent.sh | bash`
- [Install: MLX LM](https://insidethemachine.ai/install/mlx-lm.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/mlx-lm.sh | bash`
- [Install: Ollama](https://insidethemachine.ai/install/ollama.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/ollama.sh | bash`
- [Install: llama.cpp](https://insidethemachine.ai/install/llama-cpp.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/llama-cpp.sh | bash`
- [Install: LM Studio](https://insidethemachine.ai/install/lm-studio.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/lm-studio.sh | bash`
- [Install: Open WebUI](https://insidethemachine.ai/install/open-webui.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/open-webui.sh | bash`
- [Install: LiteLLM](https://insidethemachine.ai/install/litellm.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/litellm.sh | bash`
- [Install: Aider](https://insidethemachine.ai/install/aider.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/aider.sh | bash`
- [Install: Hermes Agent](https://insidethemachine.ai/install/hermes-agent.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/hermes-agent.sh | bash`
- [Install: Mem0](https://insidethemachine.ai/install/mem0.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/mem0.sh | bash`
- [Install: MCP Servers (official)](https://insidethemachine.ai/install/mcp-servers.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/mcp-servers.sh | bash`
- [Install: whisper.cpp](https://insidethemachine.ai/install/whisper-cpp.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/whisper-cpp.sh | bash`
- [Install: Docling](https://insidethemachine.ai/install/docling.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/docling.sh | bash`
- [Install: AGENTS.md](https://insidethemachine.ai/install/agents-md.md) [docs-checked]: `curl -fsSL https://insidethemachine.ai/install/agents-md.sh | bash`
- [What changed](https://insidethemachine.ai/changes.md): dated new models (with the memory they need), tools, machines, and what broke. JSON: https://insidethemachine.ai/changes.json · RSS: https://insidethemachine.ai/changes.xml

## Pages (every page also answers Accept: text/markdown)

- [Start here: local AI in plain words](https://insidethemachine.ai/start-here.md): primer + recipe list + the full runbook. JSON: https://insidethemachine.ai/start-here.json
- [Recipes](https://insidethemachine.ai/do.md): 6 goals with memory needs. JSON: https://insidethemachine.ai/do.json
- [Recipe: A private ChatGPT](https://insidethemachine.ai/do/chat.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/chat.json
- [Recipe: A coding helper](https://insidethemachine.ai/do/code.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/code.json
- [Recipe: Chat with your files](https://insidethemachine.ai/do/docs.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/docs.json
- [Recipe: Speech to text](https://insidethemachine.ai/do/voice.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/voice.json
- [Recipe: Make images](https://insidethemachine.ai/do/images.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/images.json
- [Recipe: An agent that does tasks](https://insidethemachine.ai/do/agent.md): steps, model per memory size, rollback, fixes. JSON: https://insidethemachine.ai/do/agent.json
- [Harnesses](https://insidethemachine.ai/harnesses.md): which app for which job, Claude Code vs Codex with local models. JSON: https://insidethemachine.ai/harnesses.json
- [Advanced: an agent that improves itself](https://insidethemachine.ai/guides/agent.md): same as start.md. JSON: https://insidethemachine.ai/guides/agent.json
- [Gap Score](https://insidethemachine.ai/gap-score.md): methodology, no scores yet. JSON: https://insidethemachine.ai/gap-score.json

## Core

- [Catalog JSON](https://insidethemachine.ai/api/index.json): every listing with links
- [Listing JSON Schema](https://insidethemachine.ai/schema/listing.schema.json)
- [Full text](https://insidethemachine.ai/llms-full.txt)
- [Dropped / archived items](https://insidethemachine.ai/api/dropped.json)
- [About / Gap Score methodology](https://insidethemachine.ai/about.md)

## Runtimes & inference

- [Ollama](https://insidethemachine.ai/setups/ollama.md): One command to download and run a model. [runbook]
- [SGLang](https://insidethemachine.ai/setups/sglang.md): High-throughput serving for LLMs and VLMs.
- [MLX](https://insidethemachine.ai/setups/mlx.md): Apple's array framework for Apple silicon.
- [llamafile](https://insidethemachine.ai/setups/llamafile.md): Package a model and runtime into one executable.
- [MLC LLM](https://insidethemachine.ai/setups/mlc-llm.md): Compile and run LLMs on any device.
- [KoboldCpp](https://insidethemachine.ai/setups/koboldcpp.md): One-file GGUF runner with a built-in UI.
- [MLX Examples](https://insidethemachine.ai/setups/mlx-examples.md): Official MLX example models and scripts.
- [MLX LM](https://insidethemachine.ai/setups/mlx-lm.md): Runs LLMs natively on Apple silicon. [runbook]
- [llama-swap](https://insidethemachine.ai/setups/llama-swap.md): Hot-swaps models behind one OpenAI-style endpoint.
- [MLX-VLM](https://insidethemachine.ai/setups/mlx-vlm.md): Vision-language models on MLX.
- [LM Studio](https://insidethemachine.ai/setups/lm-studio.md): A desktop app for local models. [runbook]
- [RamaLama](https://insidethemachine.ai/setups/ramalama.md): Serve models from any source in containers.
- [vLLM Metal](https://insidethemachine.ai/setups/vllm-metal.md): vLLM plugin for Apple Silicon.
- [MLX OpenAI Server](https://insidethemachine.ai/setups/mlx-openai-server.md): OpenAI-compatible API server for MLX models.
- [ML Drift](https://insidethemachine.ai/setups/ml-drift.md): Google's GPU inference engine for edge devices.
- [llama.cpp](https://insidethemachine.ai/setups/llama-cpp.md): The engine behind most local AI. [runbook]
- [vLLM](https://insidethemachine.ai/setups/vllm.md): High-throughput LLM serving engine.

## Model hubs

- [Transformers](https://insidethemachine.ai/setups/transformers.md): The reference library for open model weights.
- [Unsloth notebooks](https://insidethemachine.ai/setups/unsloth-notebooks.md): 250+ ready fine-tuning notebooks.
- [Hugging Face Hub](https://insidethemachine.ai/setups/hugging-face-hub.md): The main hub for open model weights.
- [Ollama Library](https://insidethemachine.ai/setups/ollama-library.md): Ready-to-pull models for Ollama.
- [LM Studio model catalog](https://insidethemachine.ai/setups/lm-studio-models.md): Curated models you can load in LM Studio.
- [mlx-community (Hugging Face)](https://insidethemachine.ai/setups/mlx-community.md): Community MLX conversions of popular models.
- [Unsloth (Hugging Face)](https://insidethemachine.ai/setups/unsloth-models.md): Unsloth GGUF and fine-tune-ready model uploads.
- [lmstudio-community (Hugging Face)](https://insidethemachine.ai/setups/lmstudio-community.md): LM Studio's community GGUF/MLX uploads.
- [Kaggle Models](https://insidethemachine.ai/setups/kaggle-models.md): Google-run model hub with downloadable weights.

## Models from the channel

- [Qwen-Image-2.1-Turbo](https://insidethemachine.ai/setups/qwen-image-2-1-turbo.md): 7B image model that paints and edits in 8 steps.
- [Mellum2.1 12B-A2.5B](https://insidethemachine.ai/setups/mellum-2-1.md): JetBrains coding model, 12B with 2.5B active.
- [EmbeddingGemma 2](https://insidethemachine.ai/setups/embeddinggemma-2.md): 740M on-device embedding model for search.
- [Kolibri-1](https://insidethemachine.ai/setups/kolibri-1.md): Open German/English MoE model (about 78 GB of memory, per the video).
- [AstaBrief 8B](https://insidethemachine.ai/setups/astabrief-8b.md): Small Ai2 model that writes cited reports.
- [Clef](https://insidethemachine.ai/setups/cloudflare-clef.md): Cloudflare's small open decision models (Clef, Clef-flash).

## Coding agents

- [OpenCode](https://insidethemachine.ai/setups/opencode.md): Open-source coding agent for your terminal.
- [Pi (pi-mono)](https://insidethemachine.ai/setups/pi-pi-mono.md): Small agent toolkit: LLM API, agent loop and coding CLI.
- [Goose](https://insidethemachine.ai/setups/goose.md): Extensible open-source agent that runs code and tools.
- [Aider](https://insidethemachine.ai/setups/aider.md): A coding agent in your terminal. [runbook]
- [mini-SWE-agent](https://insidethemachine.ai/setups/mini-swe-agent.md): A 100-line agent that fixes GitHub issues.
- [Claude Code](https://insidethemachine.ai/setups/claude-code.md): Anthropic's terminal coding agent (cloud model).
- [Gemini CLI](https://insidethemachine.ai/setups/gemini-cli.md): Google's open-source terminal agent for Gemini.
- [OpenHands](https://insidethemachine.ai/setups/openhands.md): Open platform for autonomous software agents.
- [Cline](https://insidethemachine.ai/setups/cline.md): Coding agent as a VS Code extension, CLI or SDK.
- [Continue](https://insidethemachine.ai/setups/continue.md): Open-source coding agent for IDEs and CI.
- [Tabby](https://insidethemachine.ai/setups/tabby.md): Self-hosted code completion and chat server.
- [Crush](https://insidethemachine.ai/setups/crush.md): Polished terminal coding agent from Charm.
- [Qwen Code](https://insidethemachine.ai/setups/qwen-code.md): Qwen's open-source terminal coding agent.
- [Kilo Code](https://insidethemachine.ai/setups/kilo-code.md): All-in-one agentic coding extension and CLI.
- [Codex CLI](https://insidethemachine.ai/setups/codex-cli.md): OpenAI's lightweight terminal coding agent.
- [GitHub Copilot CLI](https://insidethemachine.ai/setups/copilot-cli.md): GitHub's terminal coding agent; can use Ollama models.

## General & personal agents

- [OpenClaw](https://insidethemachine.ai/setups/openclaw.md): Personal agent that runs on your machine and chats anywhere.
- [Hermes Agent](https://insidethemachine.ai/setups/hermes-agent.md): A personal agent that learns skills and remembers you. [runbook]
- [AutoGPT](https://insidethemachine.ai/setups/autogpt.md): Platform for building autonomous agents.
- [Browser Use](https://insidethemachine.ai/setups/browser-use.md): Let an AI agent drive a web browser.
- [DeerFlow](https://insidethemachine.ai/setups/deerflow.md): Long-running research and coding super-agent.
- [Open Interpreter](https://insidethemachine.ai/setups/open-interpreter.md): Agent that runs code on your computer, for open models.
- [OpenManus](https://insidethemachine.ai/setups/openmanus.md): Open-source general agent (Manus-style).
- [nanobot](https://insidethemachine.ai/setups/nanobot.md): Ultra-light self-hosted personal agent.
- [UI-TARS Desktop](https://insidethemachine.ai/setups/ui-tars-desktop.md): Multimodal agent that operates your desktop.
- [Khoj](https://insidethemachine.ai/setups/khoj.md): Self-hostable AI second brain over your docs and the web.
- [Perplexica](https://insidethemachine.ai/setups/perplexica.md): Open answering engine (formerly Perplexica).
- [NanoClaw](https://insidethemachine.ai/setups/nanoclaw.md): Lightweight, container-sandboxed OpenClaw alternative.
- [GPT Researcher](https://insidethemachine.ai/setups/gpt-researcher.md): Agent that writes deep research reports.
- [Cua](https://insidethemachine.ai/setups/cua.md): Open-source computer-use drivers and sandboxes.
- [LM Studio Bionic](https://insidethemachine.ai/setups/lm-studio-bionic.md): LM Studio's agent that can run beside the app.

## Agent frameworks

- [LiteLLM](https://insidethemachine.ai/setups/litellm.md): One API in front of every model, local or cloud. [runbook]
- [AutoGen](https://insidethemachine.ai/setups/autogen.md): Microsoft's multi-agent programming framework.
- [CrewAI](https://insidethemachine.ai/setups/crewai.md): Role-based multi-agent framework.
- [LangGraph](https://insidethemachine.ai/setups/langgraph.md): Build stateful, resilient agent graphs.
- [Agno](https://insidethemachine.ai/setups/agno.md): Build and run agent systems in Python.
- [DSPy](https://insidethemachine.ai/setups/dspy.md): Program LLMs instead of hand-writing prompts.
- [OpenAI Agents SDK](https://insidethemachine.ai/setups/openai-agents-sdk.md): OpenAI's lightweight multi-agent SDK.
- [smolagents](https://insidethemachine.ai/setups/smolagents.md): Tiny library for agents that think in code.
- [Mastra](https://insidethemachine.ai/setups/mastra.md): TypeScript framework for agents and workflows.
- [Vercel AI SDK](https://insidethemachine.ai/setups/vercel-ai-sdk.md): TypeScript toolkit for AI apps and agents.
- [Haystack](https://insidethemachine.ai/setups/haystack.md): Orchestration framework for RAG and agents.
- [Letta](https://insidethemachine.ai/setups/letta.md): Stateful agents with long-term memory (ex-MemGPT).
- [Google ADK](https://insidethemachine.ai/setups/google-adk.md): Google's code-first agent development kit.
- [Pydantic AI](https://insidethemachine.ai/setups/pydantic-ai.md): Typed Python agent framework.

## Memory

- [Mem0](https://insidethemachine.ai/setups/mem0.md): Long-term memory for your agent. [runbook]
- [claude-mem](https://insidethemachine.ai/setups/claude-mem.md): Captures agent sessions and replays context later.
- [Cognee](https://insidethemachine.ai/setups/cognee.md): Knowledge-graph memory for agents.
- [Graphiti](https://insidethemachine.ai/setups/graphiti.md): Real-time temporal knowledge graphs for agents.
- [Supermemory](https://insidethemachine.ai/setups/supermemory.md): Fast memory engine and app, can run locally.
- [MemOS](https://insidethemachine.ai/setups/memos.md): Memory operating system for LLM agents.
- [Honcho](https://insidethemachine.ai/setups/honcho.md): Memory library that models users across sessions.
- [Basic Memory](https://insidethemachine.ai/setups/basic-memory.md): Markdown-file memory your agent reads and writes.
- [Memobase](https://insidethemachine.ai/setups/memobase.md): User-profile memory for chat apps.
- [memsearch](https://insidethemachine.ai/setups/memsearch.md): Semantic search over agent memory files.
- [MCP Memory Service](https://insidethemachine.ai/setups/mcp-memory-service.md): Persistent memory exposed as an MCP server.

## RAG & document parsing

- [Docling](https://insidethemachine.ai/setups/docling.md): Turns documents into clean text for AI. [runbook]
- [Marker](https://insidethemachine.ai/setups/marker.md): Fast, accurate PDF-to-markdown converter.
- [LightRAG](https://insidethemachine.ai/setups/lightrag.md): Simple, fast graph-based RAG.
- [Qdrant](https://insidethemachine.ai/setups/qdrant.md): High-performance vector database.
- [Chroma](https://insidethemachine.ai/setups/chroma.md): Open-source embedding database.
- [pgvector](https://insidethemachine.ai/setups/pgvector.md): Vector search inside Postgres.
- [LanceDB](https://insidethemachine.ai/setups/lancedb.md): Embedded multimodal vector database.
- [sqlite-vec](https://insidethemachine.ai/setups/sqlite-vec.md): Vector search as a tiny SQLite extension.
- [MarkItDown](https://insidethemachine.ai/setups/markitdown.md): Convert files and Office docs to markdown.
- [RAGFlow](https://insidethemachine.ai/setups/ragflow.md): Deep-document RAG engine with a UI.
- [PaddleOCR](https://insidethemachine.ai/setups/paddleocr.md): OCR and document parsing toolkit.
- [MinerU](https://insidethemachine.ai/setups/mineru.md): Converts complex PDFs to markdown or JSON.
- [AnythingLLM](https://insidethemachine.ai/setups/anythingllm.md): All-in-one desktop app for chat with your docs.
- [Meilisearch](https://insidethemachine.ai/setups/meilisearch.md): Fast search engine with hybrid vector search.

## Routing & gateways

- [Strands Decider](https://insidethemachine.ai/setups/strands-decider.md): Small decision model that routes agent choices.
- [New API](https://insidethemachine.ai/setups/new-api.md): Unified gateway that converts many LLM APIs.
- [One API](https://insidethemachine.ai/setups/one-api.md): LLM API management and distribution gateway.
- [Portkey Gateway](https://insidethemachine.ai/setups/portkey-gateway.md): Fast AI gateway with guardrails and fallbacks.
- [Bifrost](https://insidethemachine.ai/setups/bifrost.md): High-performance LLM and MCP gateway.
- [Plano](https://insidethemachine.ai/setups/plano.md): AI-native proxy with smart LLM routing.
- [LLMRouter](https://insidethemachine.ai/setups/llmrouter.md): Research library of LLM routing methods.
- [Envoy AI Gateway](https://insidethemachine.ai/setups/envoy-ai-gateway.md): Envoy-based gateway for generative AI services.
- [NVIDIA LLM Router](https://insidethemachine.ai/setups/nvidia-llm-router.md): Routes each prompt to the best-fit model.
- [NVIDIA PAIR](https://insidethemachine.ai/setups/nvidia-pair.md): Shares AI requests across the computers on your home network.

## MCP servers & tools

- [MCP Servers (official)](https://insidethemachine.ai/setups/mcp-servers.md): Standard plug-in tools for agents. [runbook]
- [MCP Python SDK](https://insidethemachine.ai/setups/mcp-python-sdk.md): Official Python SDK for MCP servers and clients.
- [LLM CLI](https://insidethemachine.ai/setups/llm-cli.md): CLI and Python library for any LLM, local or cloud.
- [MCP Inspector](https://insidethemachine.ai/setups/mcp-inspector.md): Visual tester for MCP servers.
- [n8n](https://insidethemachine.ai/setups/n8n.md): Workflow automation with AI nodes, self-hostable.
- [Firecrawl](https://insidethemachine.ai/setups/firecrawl.md): Turn websites into LLM-ready data.
- [Awesome MCP Servers](https://insidethemachine.ai/setups/awesome-mcp-servers.md): Big curated list of MCP servers.
- [Crawl4AI](https://insidethemachine.ai/setups/crawl4ai.md): Open-source crawler that outputs clean markdown.
- [Context7](https://insidethemachine.ai/setups/context7.md): Up-to-date library docs for coding agents.
- [Chrome DevTools MCP](https://insidethemachine.ai/setups/chrome-devtools-mcp.md): Chrome DevTools as an MCP server.
- [SearXNG](https://insidethemachine.ai/setups/searxng.md): Self-hosted private metasearch engine.
- [Playwright MCP](https://insidethemachine.ai/setups/playwright-mcp.md): Browser automation MCP server.
- [GitHub MCP Server](https://insidethemachine.ai/setups/github-mcp-server.md): GitHub's official MCP server.
- [Serena](https://insidethemachine.ai/setups/serena.md): Semantic code retrieval and editing over MCP.

## Chat UIs & front-ends

- [Open WebUI](https://insidethemachine.ai/setups/open-webui.md): A ChatGPT-style app for your local models. [runbook]
- [NextChat](https://insidethemachine.ai/setups/nextchat.md): Lightweight cross-platform chat client.
- [LobeChat](https://insidethemachine.ai/setups/lobechat.md): Agent workspace and chat UI.
- [Cherry Studio](https://insidethemachine.ai/setups/cherry-studio.md): Desktop AI studio for many providers.
- [text-generation-webui](https://insidethemachine.ai/setups/text-generation-webui.md): Desktop app for local LLMs (text-generation-webui).
- [LibreChat](https://insidethemachine.ai/setups/librechat.md): Self-hosted multi-provider ChatGPT clone.
- [Jan](https://insidethemachine.ai/setups/jan.md): Offline ChatGPT alternative for your desktop.
- [Chatbox](https://insidethemachine.ai/setups/chatbox.md): Desktop and mobile AI client.
- [SillyTavern](https://insidethemachine.ai/setups/sillytavern.md): Power-user chat and roleplay front-end.
- [Open-LLM-VTuber](https://insidethemachine.ai/setups/open-llm-vtuber.md): Voice chat with an animated avatar, locally.
- [HF Chat UI](https://insidethemachine.ai/setups/hf-chat-ui.md): The open code behind HuggingChat.
- [big-AGI](https://insidethemachine.ai/setups/big-agi.md): Advanced multi-model chat suite.
- [Enchanted](https://insidethemachine.ai/setups/enchanted.md): Native macOS/iOS app for self-hosted models.
- [llama.vscode](https://insidethemachine.ai/setups/llama-vscode.md): Local code completion in VS Code via llama.cpp.

## Voice: speech-to-text & TTS

- [whisper.cpp](https://insidethemachine.ai/setups/whisper-cpp.md): Speech-to-text that runs on your machine. [runbook]
- [MLX-Audio](https://insidethemachine.ai/setups/mlx-audio.md): TTS, STT and speech-to-speech on MLX.
- [WhisperKit](https://insidethemachine.ai/setups/whisperkit.md): On-device speech AI for Apple Silicon (WhisperKit).
- [Parakeet MLX](https://insidethemachine.ai/setups/parakeet-mlx.md): NVIDIA Parakeet speech recognition on MLX.
- [Whisper](https://insidethemachine.ai/setups/whisper.md): OpenAI's original speech recognition model.
- [ChatTTS](https://insidethemachine.ai/setups/chattts.md): Conversational text-to-speech model.
- [Handy](https://insidethemachine.ai/setups/handy.md): Offline push-to-talk dictation app.
- [Fish Speech](https://insidethemachine.ai/setups/fish-speech.md): Open-source text-to-speech with voice cloning.
- [Chatterbox](https://insidethemachine.ai/setups/chatterbox.md): Open-source expressive TTS.
- [faster-whisper](https://insidethemachine.ai/setups/faster-whisper.md): Whisper transcription, faster, via CTranslate2.
- [WhisperX](https://insidethemachine.ai/setups/whisperx.md): Whisper with word timestamps and speaker labels.
- [CosyVoice](https://insidethemachine.ai/setups/cosyvoice.md): Multilingual voice generation model.
- [Dia](https://insidethemachine.ai/setups/dia.md): TTS that generates realistic dialogue.
- [NeMo (Parakeet)](https://insidethemachine.ai/setups/nemo-parakeet.md): NVIDIA NeMo speech models (incl. Parakeet).

## Image & video generation

- [stable-diffusion.cpp](https://insidethemachine.ai/setups/stable-diffusion-cpp.md): Image and video diffusion in pure C/C++.
- [mflux](https://insidethemachine.ai/setups/mflux.md): MLX-native image and video models for Mac.
- [Qwen-Image 2.1](https://insidethemachine.ai/setups/qwen-image-2-1.md): Qwen's open image generation and editing model.
- [Draw Things](https://insidethemachine.ai/setups/draw-things.md): Community repo for the Draw Things Mac/iOS app.
- [Stable Diffusion WebUI](https://insidethemachine.ai/setups/stable-diffusion-webui.md): The classic Stable Diffusion web UI.
- [ComfyUI](https://insidethemachine.ai/setups/comfyui.md): Node-based GUI and backend for diffusion models.
- [Fooocus](https://insidethemachine.ai/setups/fooocus.md): Prompt-first, low-setup image generator.
- [Diffusers](https://insidethemachine.ai/setups/diffusers.md): PyTorch library for image, video and audio diffusion.
- [Open-Sora](https://insidethemachine.ai/setups/open-sora.md): Open video generation models.
- [InvokeAI](https://insidethemachine.ai/setups/invokeai.md): Professional creative studio for diffusion.
- [Core ML Stable Diffusion](https://insidethemachine.ai/setups/core-ml-stable-diffusion.md): Stable Diffusion on Core ML for Apple Silicon.
- [Wan 2.2](https://insidethemachine.ai/setups/wan-2-2.md): Open video generation models (Wan 2.2).
- [Wan 2.1](https://insidethemachine.ai/setups/wan-2-1.md): Open video generation models (Wan 2.1).
- [ComfyUI Manager](https://insidethemachine.ai/setups/comfyui-manager.md): Install and manage ComfyUI custom nodes.

## Fine-tuning

- [MLX-LM-LoRA](https://insidethemachine.ai/setups/mlx-lm-lora.md): LoRA and full fine-tuning on MLX.
- [Unsloth](https://insidethemachine.ai/setups/unsloth.md): Fast, low-memory fine-tuning with a local UI.
- [LLaMA-Factory](https://insidethemachine.ai/setups/llama-factory.md): Fine-tune 100+ LLMs and VLMs from one tool.
- [nanoGPT](https://insidethemachine.ai/setups/nanogpt.md): The simplest repo to train small GPTs.
- [nanochat](https://insidethemachine.ai/setups/nanochat.md): Train a small ChatGPT end to end on a budget.
- [Open R1](https://insidethemachine.ai/setups/open-r1.md): Open reproduction of DeepSeek-R1 training.
- [verl](https://insidethemachine.ai/setups/verl.md): Reinforcement-learning post-training framework.
- [PEFT](https://insidethemachine.ai/setups/peft.md): LoRA and other parameter-efficient fine-tuning.
- [TRL](https://insidethemachine.ai/setups/trl.md): Train models with SFT, DPO and RL.
- [ms-swift](https://insidethemachine.ai/setups/ms-swift.md): Fine-tune 600+ LLMs with PEFT or full training.
- [Axolotl](https://insidethemachine.ai/setups/axolotl.md): Config-driven fine-tuning toolkit.
- [OpenRLHF](https://insidethemachine.ai/setups/openrlhf.md): Scalable RLHF and agentic RL on Ray.
- [Oumi](https://insidethemachine.ai/setups/oumi.md): Fine-tune, evaluate and deploy open models.
- [MergeKit](https://insidethemachine.ai/setups/mergekit.md): Merge model weights into new models.

## Quantization

- [bitsandbytes](https://insidethemachine.ai/setups/bitsandbytes.md): 4-bit and 8-bit quantization for PyTorch.
- [Core ML Tools](https://insidethemachine.ai/setups/core-ml-tools.md): Convert and compress models for Core ML.
- [TensorRT Model Optimizer](https://insidethemachine.ai/setups/tensorrt-model-optimizer.md): NVIDIA quantization, pruning and distillation.
- [LLM Compressor](https://insidethemachine.ai/setups/llm-compressor.md): Compress models for vLLM serving.
- [Optimum](https://insidethemachine.ai/setups/optimum.md): Hardware-optimized inference and quantization for HF models.
- [torchao](https://insidethemachine.ai/setups/torchao.md): PyTorch-native quantization.
- [ExLlamaV3](https://insidethemachine.ai/setups/exllamav3.md): Fast EXL3 quantization and inference on consumer GPUs.
- [AutoRound](https://insidethemachine.ai/setups/autoround.md): Accurate low-bit post-training quantization.
- [AQLM](https://insidethemachine.ai/setups/aqlm.md): Extreme compression with additive quantization.
- [GPTQModel](https://insidethemachine.ai/setups/gptqmodel.md): GPTQ quantization toolkit across GPU vendors.
- [Optimum Quanto](https://insidethemachine.ai/setups/optimum-quanto.md): Simple PyTorch quantization backend.

## Evals & benchmarks

- [ThinkingBox](https://insidethemachine.ai/setups/thinkingbox.md): Runs agents against MCP tools and checks what they really did.
- [promptfoo](https://insidethemachine.ai/setups/promptfoo.md): Test and red-team prompts, agents and RAG.
- [OpenAI Evals](https://insidethemachine.ai/setups/openai-evals.md): Framework and registry for evaluating LLMs.
- [DeepEval](https://insidethemachine.ai/setups/deepeval.md): Unit-test style evaluation for LLM apps.
- [Ragas](https://insidethemachine.ai/setups/ragas.md): Metrics for evaluating RAG pipelines.
- [LM Evaluation Harness](https://insidethemachine.ai/setups/lm-evaluation-harness.md): The standard harness for academic LLM benchmarks.
- [Gorilla / BFCL](https://insidethemachine.ai/setups/gorilla-bfcl.md): Function-calling models and the Berkeley tool-use leaderboard.
- [OpenCompass](https://insidethemachine.ai/setups/opencompass.md): Large evaluation platform across many benchmarks.
- [SWE-bench](https://insidethemachine.ai/setups/swe-bench.md): Benchmark of real GitHub issues for coding agents.
- [simple-evals](https://insidethemachine.ai/setups/simple-evals.md): OpenAI's lightweight reference evals.
- [LMMs-Eval](https://insidethemachine.ai/setups/lmms-eval.md): Evaluation toolkit for multimodal models.
- [Inspect AI](https://insidethemachine.ai/setups/inspect-ai.md): UK AISI framework for LLM and agent evals.
- [Terminal-Bench](https://insidethemachine.ai/setups/terminal-bench.md): Benchmark of hard tasks in a real terminal.
- [LightEval](https://insidethemachine.ai/setups/lighteval.md): Hugging Face's multi-backend eval toolkit.

## Scaffolds & templates

- [AGENTS.md](https://insidethemachine.ai/setups/agents-md.md): A README written for AI agents. [runbook]
- [ClawHub](https://insidethemachine.ai/setups/clawhub.md): Skill and plugin registry for OpenClaw.
- [Superpowers](https://insidethemachine.ai/setups/superpowers.md): Agent skills library and dev methodology.
- [Anthropic Skills](https://insidethemachine.ai/setups/anthropic-skills.md): Anthropic's public Agent Skills examples.
- [Spec Kit](https://insidethemachine.ai/setups/spec-kit.md): Spec-driven development toolkit for agents.
- [OpenSpec](https://insidethemachine.ai/setups/openspec.md): Lightweight spec-driven workflow for coding agents.
- [Awesome Claude Code](https://insidethemachine.ai/setups/awesome-claude-code.md): Curated Claude Code commands, hooks and workflows.
- [BMAD Method](https://insidethemachine.ai/setups/bmad-method.md): Agile multi-agent development method.
- [Awesome Cursor Rules](https://insidethemachine.ai/setups/awesome-cursor-rules.md): Collection of agent rule files.
- [wshobson agents](https://insidethemachine.ai/setups/wshobson-agents.md): Multi-harness agent plugin marketplace.
- [Claude Code Templates](https://insidethemachine.ai/setups/claude-code-templates.md): CLI to install agent configs and templates.
- [Vercel Agent Skills](https://insidethemachine.ai/setups/vercel-agent-skills.md): Vercel's official agent skills.
- [Task Master](https://insidethemachine.ai/setups/task-master.md): Task-management system for coding agents.
- [SOUL.md (persona file)](https://insidethemachine.ai/setups/soul-md.md): A file that sets your agent's identity and voice.

## Observability

- [Langfuse](https://insidethemachine.ai/setups/langfuse.md): Open-source tracing and evals for LLM apps.
- [MLflow](https://insidethemachine.ai/setups/mlflow.md): ML and LLM lifecycle platform with tracing.
- [Opik](https://insidethemachine.ai/setups/opik.md): Trace, evaluate and monitor LLM apps.
- [Phoenix](https://insidethemachine.ai/setups/phoenix.md): Open-source LLM tracing and evaluation.
- [Evidently](https://insidethemachine.ai/setups/evidently.md): Evaluate and monitor ML and LLM systems.
- [OpenLLMetry](https://insidethemachine.ai/setups/openllmetry.md): OpenTelemetry instrumentation for LLM apps.
- [Helicone](https://insidethemachine.ai/setups/helicone.md): LLM observability via a one-line proxy.
- [AgentOps](https://insidethemachine.ai/setups/agentops.md): Monitoring and cost tracking for agents.
- [Agenta](https://insidethemachine.ai/setups/agenta.md): Prompt management, evals and observability.
- [Logfire](https://insidethemachine.ai/setups/logfire.md): Observability for LLM and agent systems.
- [OpenLIT](https://insidethemachine.ai/setups/openlit.md): OpenTelemetry-native tracing and guardrails.
- [Weave](https://insidethemachine.ai/setups/weave.md): W&B toolkit for tracing and evaluating AI apps.

## Hardware & system utilities

- [Apple container](https://insidethemachine.ai/setups/apple-container.md): Run Linux containers natively on a Mac.
- [Stats](https://insidethemachine.ai/setups/stats.md): macOS menu-bar system monitor.
- [llmfit](https://insidethemachine.ai/setups/llmfit.md): One command to see which models fit your hardware.
- [nvtop](https://insidethemachine.ai/setups/nvtop.md): GPU monitor for Apple, AMD, Intel and NVIDIA.
- [nvitop](https://insidethemachine.ai/setups/nvitop.md): Interactive NVIDIA GPU process viewer.
- [GPUStack](https://insidethemachine.ai/setups/gpustack.md): GPU cluster manager for model serving.
- [NVIDIA Container Toolkit](https://insidethemachine.ai/setups/nvidia-container-toolkit.md): Use NVIDIA GPUs inside containers.
- [mactop](https://insidethemachine.ai/setups/mactop.md): Apple Silicon CPU/GPU/power monitor in the terminal.
- [gguf-parser](https://insidethemachine.ai/setups/gguf-parser.md): Inspect GGUF files and estimate memory use.

## Optional

- [YouTube: Inside The Machine](https://www.youtube.com/@InsideTheMachine-s2u)