# Recipe: Chat with your files A private chat that answers questions about your own PDFs, notes and documents. URL: https://insidethemachine.ai/do/docs Memory: needs 16 GB+, best with 32 GB+. Platform: macOS (Apple silicon) first. Install steps run on an Apple M4 Mac with 16 GB on 2026-10-15. File upload and the first Docling run are checked against the docs only. ## Model by memory (replace {MODEL}) | RAM GB | model | ollama tag | needs GB | |---|---|---|---| | 8 | Qwen3.5 4B | qwen3.5:4b | 5.8 | | 16 | Gemma 4 12B | gemma4:12b | 11.2 | | 24 | Gemma 4 12B | gemma4:12b | 11.2 | | 32 | Qwen3.8 27B | qwen3.8:27b | 22.7 | | 48 | Qwen3.8 27B | qwen3.8:27b | 22.7 | | 64 | Qwen3.8 27B | qwen3.8:27b | 22.7 | | 96 | Qwen3.8 27B | qwen3.8:27b | 22.7 | | 128 | Qwen3.8 27B | qwen3.8:27b | 22.7 | | 192 | Qwen3.8-Flash-Next (125B-A6B) | qwen3.8-flash-next:125b-a6b | 140 | | 256 | Qwen3.8-Flash-Next (125B-A6B) | qwen3.8-flash-next:125b-a6b | 140 | | 512 | Qwen3.8-Flash-Next (125B-A6B) | qwen3.8-flash-next:125b-a6b | 140 | ## Steps 1. **Install Ollama**: Ollama downloads AI models and runs them on your computer. Open the Terminal app and paste this. It may ask for your Mac password. ```sh curl -fsSL https://ollama.com/install.sh | sh ``` (Or download the app from ollama.com/download) Expect: Run `ollama --version`. It prints a version number. If it also says “could not connect”, open Ollama from your Applications folder, wait a few seconds, then go on. 2. **Download your model**: This is the AI itself. It is one big file, so it takes a few minutes. ```sh ollama pull {MODEL} ``` Expect: The download finishes with “success”. 3. **Add a ChatGPT-style window**: Open WebUI gives you a chat app in your browser. It needs Homebrew (brew.sh) for Python. ```sh brew install python@3.11 ``` ```sh python3.11 -m venv ~/.venvs/openwebui && source ~/.venvs/openwebui/bin/activate ``` ```sh pip install open-webui ``` ```sh open-webui serve ``` (Keep this window open while you use it) Expect: Open http://localhost:8080. Make a local account (it stays on your computer) and pick your model at the top. 4. **Add your files**: In a chat, click + and attach a file, then ask about it. For a whole folder, go to Workspace → Knowledge, make a collection and upload your files. Then type # in a chat to use it. Expect: The answer quotes your document. 5. **Messy PDF? Clean it first**: Scanned or complex PDFs give bad answers. Docling turns them into clean text you can upload instead. ```sh python3 -m venv ~/.venvs/docling && source ~/.venvs/docling/bin/activate ``` ```sh pip install docling ``` ```sh docling yourfile.pdf ``` (Writes yourfile.md next to it) Expect: A Markdown file appears. Upload that one. ## Alternative Prefer one app? Use AnythingLLM. AnythingLLM is a desktop app for chatting with your documents. Download it from anythingllm.com and choose Ollama as the model provider. (https://insidethemachine.ai/setups/anythingllm) ## Rollback - `ollama rm {MODEL}` (Removes the model file) - `rm -rf ~/.venvs/openwebui ~/.venvs/docling` (Removes Open WebUI and Docling) - `rm -rf ~/Library/Caches/pip` (Removes saved downloads) ## One-command install ```bash curl -fsSL https://insidethemachine.ai/install/docs.sh | bash # install, then check curl -fsSL https://insidethemachine.ai/install/docs.sh | bash -s -- plan # print only curl -fsSL https://insidethemachine.ai/install/docs.sh | bash -s -- rollback --yes ``` Run status and Windows: https://insidethemachine.ai/install/docs.md. The script picks {MODEL} from the machine's memory (override: ITM_RAM, ITM_MODEL). ## Notes - Check your free disk space first. You need room for the model plus about 4 GB for Open WebUI and 2 GB for Docling. - The first Docling run downloads its own small models. - Answers are only as good as the text it can read. Clean files give better answers. ## If something goes wrong - “could not connect to ollama app, is it running?”: Ollama is not running. Open Ollama from your Applications folder, or run `ollama serve` in a new Terminal window and leave it open. - “command not found: ollama”: Close the Terminal window and open a new one. If it still fails, run the install command again. - “model requires more system memory”: The model is too big for your memory. Pick a smaller memory size at the top of this page and use that model. Close other big apps too. - “file does not exist” when you download: The model name is wrong, or Ollama is too old. Copy the command again from this page. Then update Ollama: run the install command again. - It is very slow: Run `ollama ps`. PROCESSOR should say 100% GPU. If it shows CPU, the model does not fit in memory. Pick a smaller model. - The computer freezes, or the fan is loud: The model uses all your memory. Stop it with `ollama stop {MODEL}`. Then use a smaller one. - “address already in use” when Open WebUI starts: Another app uses port 8080. Run `open-webui serve --port 8081`, then open http://localhost:8081. - No models in the list at the top of Open WebUI: Ollama is not running. Start it, then reload the page. - The answer ignores your file: Check that the file shows in the chat. Scanned PDFs have no real text; clean them with Docling (step 5) and upload the .md file. ## After it works - Try it: upload a manual and ask “how do I reset it?”. Upload notes and ask “what did we decide?”. - Make a Knowledge collection per topic, so answers only use the right files. - See every model you have: `ollama list`. - Try a different model: copy any `ollama run` name from the Models page. - Update your model: run `ollama pull {MODEL}` again. It only downloads what changed. - Free disk space: `ollama rm {MODEL}` deletes a model you no longer use. ## Windows / Linux (not tested by us) Install Ollama with this PowerShell command, or the installer from ollama.com/download. On Linux, the install command above works as it is. After that, every `ollama` command on this page is the same. Open WebUI and Docling also run on Windows; their pages have the steps. ```powershell irm https://ollama.com/install.ps1 | iex ``` (Windows: paste in PowerShell. Or get the installer from ollama.com/download)