A private ChatGPT

# Recipe: A private ChatGPT

A ChatGPT-style chat app that runs fully on your computer, even offline.

URL: https://insidethemachine.ai/do/chat
Memory: needs 8 GB+, best with 16 GB+. Platform: macOS (Apple silicon) first.
Run end to end on an Apple M4 Mac with 16 GB on 2026-10-15.

## Model by memory (replace {MODEL})

| RAM GB | model | ollama tag | needs GB |
|---|---|---|---|
| 8 | Qwen3.5 4B | qwen3.5:4b | 5.8 |
| 16 | Gemma 4 12B | gemma4:12b | 11.2 |
| 24 | Gemma 4 12B | gemma4:12b | 11.2 |
| 32 | Qwen3.8 27B | qwen3.8:27b | 22.7 |
| 48 | Qwen3.8 27B | qwen3.8:27b | 22.7 |
| 64 | Qwen3.8 27B | qwen3.8:27b | 22.7 |
| 96 | Qwen3.8 27B | qwen3.8:27b | 22.7 |
| 128 | Qwen3.8 27B | qwen3.8:27b | 22.7 |
| 192 | Qwen3.8-Flash-Next (125B-A6B) | qwen3.8-flash-next:125b-a6b | 140 |
| 256 | Qwen3.8-Flash-Next (125B-A6B) | qwen3.8-flash-next:125b-a6b | 140 |
| 512 | Qwen3.8-Flash-Next (125B-A6B) | qwen3.8-flash-next:125b-a6b | 140 |

## Steps

1. **Install Ollama**: Ollama downloads AI models and runs them on your computer. Open the Terminal app and paste this. It may ask for your Mac password.
   ```sh
   curl -fsSL https://ollama.com/install.sh | sh
   ```
   (Or download the app from ollama.com/download)
   Expect: Run `ollama --version`. It prints a version number. If it also says “could not connect”, open Ollama from your Applications folder, wait a few seconds, then go on.
2. **Download your model**: This is the AI itself. It is one big file, so it takes a few minutes.
   ```sh
   ollama pull {MODEL}
   ```
   Expect: The download finishes with “success”.
3. **Say hi**: Chat right in the Terminal to check it works.
   ```sh
   ollama run {MODEL}
   ```
   (Type a message. Type /bye to quit.)
   Expect: The model answers you. That answer came from your own computer.
4. **Add a ChatGPT-style window**: Open WebUI gives you a chat app in your browser. It needs Homebrew (brew.sh) for Python.
   ```sh
   brew install python@3.11
   ```
   ```sh
   python3.11 -m venv ~/.venvs/openwebui && source ~/.venvs/openwebui/bin/activate
   ```
   ```sh
   pip install open-webui
   ```
   ```sh
   open-webui serve
   ```
   (Keep this window open while you use it)
   Expect: Open http://localhost:8080. Make a local account (it stays on your computer) and pick your model at the top.

## Alternative

Prefer no Terminal? Use LM Studio. Download the app from lmstudio.ai, search for your model by name, click Download, then chat. Everything is point and click. (https://insidethemachine.ai/setups/lm-studio)

## Rollback

- `ollama rm {MODEL}` (Removes the model file)
- `rm -rf ~/.venvs/openwebui` (Removes Open WebUI)
- `rm -rf ~/Library/Caches/pip` (Removes about 1.5 GB of saved downloads)

## One-command install

```bash
curl -fsSL https://insidethemachine.ai/install/chat.sh | bash             # install, then check
curl -fsSL https://insidethemachine.ai/install/chat.sh | bash -s -- plan     # print only
curl -fsSL https://insidethemachine.ai/install/chat.sh | bash -s -- rollback --yes
```

Run status and Windows: https://insidethemachine.ai/install/chat.md. The script picks {MODEL} from the machine's memory (override: ITM_RAM, ITM_MODEL).

## Notes

- Check your free disk space first. You need room for the model (a bit less than the memory it needs) plus about 4 GB for Open WebUI.
- The first answer is slow while the model loads. On a 16 GB Mac it took about 1.5 minutes. After that it is quick.
- Open WebUI uses port 8080. Other local AI servers sometimes use it too.

## If something goes wrong

- “could not connect to ollama app, is it running?”: Ollama is not running. Open Ollama from your Applications folder, or run `ollama serve` in a new Terminal window and leave it open.
- “command not found: ollama”: Close the Terminal window and open a new one. If it still fails, run the install command again.
- “model requires more system memory”: The model is too big for your memory. Pick a smaller memory size at the top of this page and use that model. Close other big apps too.
- “file does not exist” when you download: The model name is wrong, or Ollama is too old. Copy the command again from this page. Then update Ollama: run the install command again.
- It is very slow: Run `ollama ps`. PROCESSOR should say 100% GPU. If it shows CPU, the model does not fit in memory. Pick a smaller model.
- The computer freezes, or the fan is loud: The model uses all your memory. Stop it with `ollama stop {MODEL}`. Then use a smaller one.
- “address already in use” when Open WebUI starts: Another app uses port 8080. Run `open-webui serve --port 8081`, then open http://localhost:8081.
- No models in the list at the top of Open WebUI: Ollama is not running. Start it, then reload the page.

## After it works

- Try it: paste a long email and ask “sum this up in 3 lines”. Ask it to rewrite a text so it sounds friendlier. Ask it to explain a hard idea step by step.
- See every model you have: `ollama list`.
- Try a different model: copy any `ollama run` name from the Models page.
- Update your model: run `ollama pull {MODEL}` again. It only downloads what changed.
- Free disk space: `ollama rm {MODEL}` deletes a model you no longer use.

## Windows / Linux (not tested by us)

Install Ollama with this PowerShell command, or the installer from ollama.com/download. On Linux, the install command above works as it is. After that, every `ollama` command on this page is the same. For the chat window on Windows, LM Studio (lmstudio.ai) is the easiest. Open WebUI also runs on Windows; see its page.

```powershell
irm https://ollama.com/install.ps1 | iex
```
(Windows: paste in PowerShell. Or get the installer from ollama.com/download)