---
slug: mlx-lm
kind: runbook
status: docs-checked
sh: https://insidethemachine.ai/install/mlx-lm.sh
ps1: null
json: https://insidethemachine.ai/install/mlx-lm.json
page: https://insidethemachine.ai/setups/mlx-lm
---

# Install: MLX LM

Apple's MLX-native way to generate, chat, fine-tune and serve LLMs on Apple silicon.

**Run status: Not run: commands checked against the official docs only.** Commands checked against the official docs on 2026-10-15. Not run end to end.

**This script:** The wrapper scripts are generated from the site data. Every .sh passed bash -n and shellcheck and ran in plan mode on Linux (Debian 13). Every .ps1 passed the PowerShell parser and chat.ps1 ran in plan, verify and rollback mode under PowerShell 7 on Linux with Windows calls mocked. Not run on macOS or real Windows.

## One command

```bash
curl -fsSL https://insidethemachine.ai/install/mlx-lm.sh | bash
```

Actions: `| bash -s -- plan` (print only), `verify`, `rollback --yes`. Env: ITM_RAM. Read the script first: `curl -fsSL https://insidethemachine.ai/install/mlx-lm.sh | less`.

Lines marked YOU RUN are not run by the script (servers, interactive apps, your own files). The script prints them at the end, in order.

## Steps

### macOS

Install:

1. **Install MLX LM**
   - run: `python3 -m venv ~/.venvs/mlx && source ~/.venvs/mlx/bin/activate`
   - run: `pip install mlx-lm`
   - note: or with conda: conda install -c conda-forge mlx-lm
2. **Configure (your choices)**
   - YOU RUN (starts an OpenAI-like server on localhost:8080; model downloads from Hugging Face on first run): `mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit`
   - YOU RUN (quantized KV cache for long contexts): `mlx_lm.server --model <hf_repo_or_path> --kv-bits 4 --prefill-step-size 512`

Verify:

- `mlx_lm.generate --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --prompt "hello"` → expect: prints generated text and tokens-per-second stats
- `curl -s http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"messages":[{"role":"user","content":"Say hi"}]}'` → expect: JSON response with a choices[0].message.content field (server must be running)

Rollback:

- run: `pip uninstall -y mlx-lm`
- run: `rm -rf ~/.venvs/mlx`
- note: downloaded models live in the Hugging Face cache (~/.cache/huggingface/hub); delete specific model folders to free space