Agents · data 2026-10-15

Best local model for agents.

Agents call tools and take many steps. Pick the model for your memory, then an agent app. Every pick comes from published scores.

Terminal-Bench 2.1 = real terminal tasks (higher is better). Claude Opus 5.5 scores 57.6 on AA.
Memory	Model	Needs	Terminal-Bench 2.1	Score (AA)	Run it
8 GB	none scored	—	—	—	—
16 GB	Mellum2.1 12B-A2.5B Thinking	11.3 GB	17.4%	—	—
24 GB	Mellum2.1 12B-A2.5B Thinking	11.3 GB	17.4%	—	—
32 GB	Qwen3.8 27B	22.7 GB	79.8%	33.7	`ollama run qwen3.8:27b`
48 GB	Qwen3.8 27B	22.7 GB	79.8%	33.7	`ollama run qwen3.8:27b`
64 GB	Qwen3.8 27B	22.7 GB	79.8%	33.7	`ollama run qwen3.8:27b`
96 GB	Qwen3.8 27B	22.7 GB	79.8%	33.7	`ollama run qwen3.8:27b`
128 GB	Qwen3.8 27B	22.7 GB	79.8%	33.7	`ollama run qwen3.8:27b`
192 GB	Qwen3.8-Flash-Next (125B-A6B)	140 GB	86.1%	39.8	`ollama run qwen3.8-flash-next:125b-a6b`
256 GB	Qwen3.8-Flash-Next (125B-A6B)	140 GB	86.1%	39.8	`ollama run qwen3.8-flash-next:125b-a6b`
512 GB	GLM-5.3-Flash	233 GB	84.3%	41.8	MLX only (Mac)

Apps

Agent apps that run local models.

messaging · MITOpenClaw

Personal assistant in your chat apps.

ollama launch openclaw

terminal · MITHermes Agent

Personal agent with a messaging gateway.

ollama launch hermes

web app · MITVane (Perplexica)

Private web search with cited answers.

In settings set the Ollama API URL to http://host.docker.internal:11434

desktop app · LM Studio Bionic

Desktop agent for documents and coding.

messaging · MITnanobot

Lightweight assistant in chat apps.

In config: "providers": { "ollama": { "apiBase": "http://localhost:11434/v1" } }

web app · Apache-2.0GPT Researcher

Deep research reports on local models.

In .env: OLLAMA_BASE_URL="http://127.0.0.1:11434/" and FAST_LLM="ollama:<model>"

Compare all apps Agent recipe

Questions

Short answers.

What is the best local model for agents on 16 GB?: Mellum2.1 12B-A2.5B Thinking (needs 11.3 GB, Terminal-Bench 2.1 17.4%).
What is the best local model for agents on 128 GB?: Qwen3.8 27B (needs 22.7 GB at 4-bit, 36.5 GB at 8-bit, Terminal-Bench 2.1 79.8%).
Is a local agent as good as one on a paid model?: Not yet. Qwen3.8 27B scores 33.7 on the Artificial Analysis index; Claude Opus 5.5 scores 57.6. That is about 59%. Good for everyday work; use a paid model for the hardest tasks.
Which agent app works with local models?: OpenClaw: Personal assistant in your chat apps (ollama launch openclaw). Hermes Agent: Personal agent with a messaging gateway (ollama launch hermes). Vane (Perplexica): Private web search with cited answers (In settings set the Ollama API URL to http://host.docker.internal:11434).

Scores are from Artificial Analysis and LMArena, not our own tests yet. Memory need = 4-bit file × 1.15 + 2 GB; a model fits when that is at most 75% of memory. Models 2026-10-15, machines 2026-10-15.

AGENT VIEW · text/markdown · or send Accept: text/markdown

# Best local model for AI agents (OpenClaw, Hermes Agent)

URL: https://insidethemachine.ai/best/local-model-for-agents
Data: models 2026-10-15, apps 2026-10-15. Picks: the site's best_agentic per memory tier in models.json.

| memory GB | model | needs GB | Terminal-Bench 2.1 | AA | run |
|---|---|---|---|---|---|
| 8 | none scored | - | - | - | - |
| 16 | Mellum2.1 12B-A2.5B Thinking | 11.3 | 17.4% | - | - |
| 24 | Mellum2.1 12B-A2.5B Thinking | 11.3 | 17.4% | - | - |
| 32 | Qwen3.8 27B | 22.7 | 79.8% | 33.7 | `ollama run qwen3.8:27b` |
| 48 | Qwen3.8 27B | 22.7 | 79.8% | 33.7 | `ollama run qwen3.8:27b` |
| 64 | Qwen3.8 27B | 22.7 | 79.8% | 33.7 | `ollama run qwen3.8:27b` |
| 96 | Qwen3.8 27B | 22.7 | 79.8% | 33.7 | `ollama run qwen3.8:27b` |
| 128 | Qwen3.8 27B | 22.7 | 79.8% | 33.7 | `ollama run qwen3.8:27b` |
| 192 | Qwen3.8-Flash-Next (125B-A6B) | 140 | 86.1% | 39.8 | `ollama run qwen3.8-flash-next:125b-a6b` |
| 256 | Qwen3.8-Flash-Next (125B-A6B) | 140 | 86.1% | 39.8 | `ollama run qwen3.8-flash-next:125b-a6b` |
| 512 | GLM-5.3-Flash | 233 | 84.3% | 41.8 | `mlx_lm.generate --model mlx-community/GLM-5.3-Flash-4bit --prompt 'hello'` |

## Apps that run them locally

- OpenClaw (messaging, MIT): Personal assistant in your chat apps. Local: `ollama launch openclaw`. Needs 64,000 context. https://openclaw.ai
- Hermes Agent (terminal, MIT): Personal agent with a messaging gateway. Local: `ollama launch hermes`. Needs 64,000 context. https://hermes-agent.nousresearch.com
- Vane (Perplexica) (web app, MIT): Private web search with cited answers. Local: In settings set the Ollama API URL to `http://host.docker.internal:11434`. https://github.com/ItzCrazyKns/Vane
- LM Studio Bionic (desktop app, null): Desktop agent for documents and coding. Local: see docs. https://lmstudio.ai/bionic
- nanobot (messaging, MIT): Lightweight assistant in chat apps. Local: In config: `"providers": { "ollama": { "apiBase": "http://localhost:11434/v1" } }`. https://nanobot.wiki
- GPT Researcher (web app, Apache-2.0): Deep research reports on local models. Local: In `.env`: `OLLAMA_BASE_URL="http://127.0.0.1:11434/"` and `FAST_LLM="ollama:<model>"`. https://gptr.dev

## Questions

**What is the best local model for agents on 16 GB?**
Mellum2.1 12B-A2.5B Thinking (needs 11.3 GB, Terminal-Bench 2.1 17.4%).

**What is the best local model for agents on 128 GB?**
Qwen3.8 27B (needs 22.7 GB at 4-bit, 36.5 GB at 8-bit, Terminal-Bench 2.1 79.8%).

**Is a local agent as good as one on a paid model?**
Not yet. Qwen3.8 27B scores 33.7 on the Artificial Analysis index; Claude Opus 5.5 scores 57.6. That is about 59%. Good for everyday work; use a paid model for the hardest tasks.

**Which agent app works with local models?**
OpenClaw: Personal assistant in your chat apps (ollama launch openclaw). Hermes Agent: Personal agent with a messaging gateway (ollama launch hermes). Vane (Perplexica): Private web search with cited answers (In settings set the Ollama API URL to http://host.docker.internal:11434).