# GeForce RTX 4090 for local AI

URL: https://insidethemachine.ai/machines/rtx-4090
Type: NVIDIA graphics card. Chip: RTX 4090. Memory: vram, options 24 GB. Bandwidth: 1008 GB/s. Spec source: https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units (2026-10-15).
Price: from about $3,099 (2026-10-02, approximate). Status: previous_gen.

## Which models fit

| memory GB | usable GB | models that fit | best 3 |
|---|---|---|---|
| 24 | 23 | 56 | Qwen3.8 27B (22.7 GB), Qwen3.5 27B (21.5 GB), Qwen3.6 27B (21.5 GB) |

## Published speeds (not our tests)

- gpt-oss-20b: 242.6 tok/s decode, 593.8 tok/s prompt (llama.cpp, Q4_K_M, RTX 4090 + EPYC 7642 (chat-short), community-measured) https://llm-speed.com/m/gpt-oss-20b (2026-10-15)

Notes: Used price is a single clean eBay sale (insufficient sample, flagged by the dataset author). Treat as very rough.

## Questions

**What is the best local LLM for the GeForce RTX 4090?**
With 24 GB, Qwen3.8 27B (needs 22.7 GB, AA 33.7). 56 of 124 models fit.

**How fast is the GeForce RTX 4090 for local AI?**
Published: gpt-oss-20b at 242.6 tokens/sec (llama.cpp, RTX 4090 + EPYC 7642 (chat-short)). These are other people's measurements, not ours.

**How much does the GeForce RTX 4090 cost?**
From about $3,099 (as of 2026-10-02). 2026 prices are inflated by the memory-chip shortage; check before buying.