---
slug: llama-cpp
kind: runbook
status: docs-checked
sh: https://insidethemachine.ai/install/llama-cpp.sh
ps1: https://insidethemachine.ai/install/llama-cpp.ps1
json: https://insidethemachine.ai/install/llama-cpp.json
page: https://insidethemachine.ai/setups/llama-cpp
---

# Install: llama.cpp

The gold-standard C/C++ inference engine for GGUF models, with Metal as a first-class backend.

**Run status: Not run: commands checked against the official docs only.** Commands checked against the official docs on 2026-10-15. Not run end to end.

**This script:** The wrapper scripts are generated from the site data. Every .sh passed bash -n and shellcheck and ran in plan mode on Linux (Debian 13). Every .ps1 passed the PowerShell parser and chat.ps1 ran in plan, verify and rollback mode under PowerShell 7 on Linux with Windows calls mocked. Not run on macOS or real Windows.

## One command

```bash
curl -fsSL https://insidethemachine.ai/install/llama-cpp.sh | bash
```

Windows (PowerShell):

```powershell
irm https://insidethemachine.ai/install/llama-cpp.ps1 | iex
```

Actions: `| bash -s -- plan` (print only), `verify`, `rollback --yes`. Env: ITM_RAM. Read the script first: `curl -fsSL https://insidethemachine.ai/install/llama-cpp.sh | less`.

Lines marked YOU RUN are not run by the script (servers, interactive apps, your own files). The script prints them at the end, in order.

## Steps

### macOS

Install:

1. **Install llama.cpp**
   - run: `curl -LsSf https://llama.app/install.sh | sh`
   - note: alternative: brew install llama.cpp   (Homebrew formula is at 0.6.0 as of 2026-10-15)
2. **Configure (your choices)**
   - YOU RUN (download from Hugging Face and chat (README example)): `llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF`
   - YOU RUN (OpenAI-compatible API server + web UI): `llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF`

Verify:

- `command -v llama` → expect: path to the llama binary (Homebrew/older builds may expose llama-cli / llama-server instead)
- `curl -s http://localhost:8080/v1/models` → expect: JSON listing the loaded model (server running; 8080 is the server's usual default, check startup log)

Rollback:

- run (macOS only): `brew uninstall llama.cpp`
- note: script install: follow uninstall instructions at https://llama.app
- note: -hf downloads are cached (macOS default ~/Library/Caches/llama.cpp); delete to free space

### Linux

Install:

1. **Install llama.cpp**
   - run: `curl -LsSf https://llama.app/install.sh | sh`
2. **Configure (your choices)**
   - YOU RUN (download from Hugging Face and chat (README example)): `llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF`
   - YOU RUN (OpenAI-compatible API server + web UI): `llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF`

Verify:

- `command -v llama` → expect: path to the llama binary (Homebrew/older builds may expose llama-cli / llama-server instead)
- `curl -s http://localhost:8080/v1/models` → expect: JSON listing the loaded model (server running; 8080 is the server's usual default, check startup log)

Rollback:

- run (macOS only): `brew uninstall llama.cpp`
- note: script install: follow uninstall instructions at https://llama.app
- note: -hf downloads are cached (macOS default ~/Library/Caches/llama.cpp); delete to free space

### Windows (PowerShell, not run by us)

Install:

1. **Install llama.cpp**
   - run: `irm https://llama.app/install.ps1 | iex`

Verify:

- by hand: `command -v llama` → expect: path to the llama binary (Homebrew/older builds may expose llama-cli / llama-server instead)
- by hand: `curl -s http://localhost:8080/v1/models` → expect: JSON listing the loaded model (server running; 8080 is the server's usual default, check startup log)

Rollback:

- YOU RUN (macOS/Linux command; do the same by hand): `brew uninstall llama.cpp`
- note: script install: follow uninstall instructions at https://llama.app
- YOU RUN (macOS/Linux command; do the same by hand): `# -hf downloads are cached (macOS default ~/Library/Caches/llama.cpp); delete to free space`