■ space · the apartment tour

THE NIGHT 4.9 BECAME 7.0

The box: Ryzen, 32 GB of RAM, an RTX 3060 with 12 GB of VRAM. The pull: llama3.1:8b, tagged 4.9 GB. Three commands, one screen:

$ ollama ps
NAME            SIZE      PROCESSOR    CONTEXT
llama3.1:8b     7.0 GB    100% GPU     32,768
$ nvidia-smi
   …  7963 MiB / 12288 MiB  …
$ free -h
Mem:  31 Gi

the capture, retyped — the pixels stayed on the bench box; the numbers are the record

A "4.9 GB model" reserved 7.0 GB before answering a single prompt — a 43% context tax — and kept 7,963 of 12,288 MiB to itself.

The weights were never the budget. The context was.

VRAM sells the dream. RAM decides whether the model boots at all — and how much context survives when it does.

THE FLOOR PLAN

Every loaded model rents two apartments at once. Pick a model class, dial the context, watch both fill. The three measured rows below anchor the dial.

rooms not to scale against each other — each flat is drawn against its own lease

WHAT THE EXTRA GIGABYTES ARE

The llama.cpp memory notes spell out the arithmetic the download pages omit: total memory = model weights + KV cache + compute buffer. Only the first term is constant.

Ollama wraps llama.cpp — same law. ollama ps: 7.0 GB for a 4.9 GB tag at a 32,000-token window, all resident on the GPU.

Read the last two rows again. On VRAM, a 14B at real context is a split-offload job; a 27B, impossible. On 32 GB of RAM, merely slow.

Impossible vs slow — the entire practical difference between RAM and VRAM. Offload rides PCIe: the Ollama FAQ is blunt — throughput drops hard as the GPU share shrinks.

Nobody chooses that trade-off consciously. It happens silently, one layer at a time — the symptom is just "local models feel overrated."

This is why the quantization ladder exists: Q4 makes the arithmetic land inside hardware people own. Two "identical" 8B setups, different context lengths, different machines entirely.

A quantized model isn't a compromise. It's an admission that memory was always the real budget.

THE HONEST LEDGER

Things that broke or surprised me while writing this piece, in order:

1. The 43% number itself. I expected the 8B model to occupy "about its file size." It reserved 7.0 GB against a 4.9 GB download. If that surprised me, it surprises everyone who reads spec sheets instead of ps output.
2. The capture session. My first screenshot grabbed the wrong window; my second worked, and that's the one the receipt block retypes. Receipts are a workflow, not a vibe — a pull → capture → ollama rm cycle keeps the disk honest.
3. What didn't break: the GPU never spilled. 8B at 32k context on 12 GB is genuinely comfortable. The card is fine. The discourse around the card is what's miscalibrated.

GPUs still matter — the 100% GPU line is why generation felt instant. But the GPU is the second question. Runner choice comes after what fits.

THE RULE OF THUMB WORTH KEEPING

RAM = model file + KV cache for your real window + 4 GB for being a computer. 7–8B Q4: 16 GB comfortable. 14B–32B: 32 GB is the point.

Buy VRAM for the speed you want at the context you use; buy RAM for everything you'll ever load.

Check ollama ps once and the spec-sheet religion quietly ends.

Two questions for your next dev box: VRAM for the benchmarks you'll post, or RAM for the models you'll run? And how many 2 a.m. pulls survived breakfast? Mine didn't.

end of the apartment tour · nothing fetched · everything local