■ space · the weighbridge · fig 1

■ the answer first

VRAM ≈ weights + KV cache + overhead; the file size is the weights line only.

An 8B at Q4_K_M, 32k context: ~9.8 GB — misses an 8 GB card by 23%. The quant was never the villain; the context was.

Reverse mode answers the real question: budget in, largest-fitting quant out. All arithmetic, in your browser, nothing uploaded.

4.85 GBthe weights line — 8 × 4.85 / 8

4.00 GBthe KV cache at 32k — 128 KB × 32,768

23%over the 8 GB card — the miss

01THE WEIGHBRIDGE

the model is the crate, the context is the spool — the red mark is the card you own

FIG 1 · THE WEIGHBRIDGEkeys 1 · 2 · 3 — or the buttons

LOAD ~9.8 GB · WEIGHTS 4.85 GB · KV 4.00 GB · TARE 0.94 GB · VERDICT MISSED BY 23%

key 2 winds 4,096 → 32,768 → 131,072 tokens onto the spool — watch the needle, not the quant

the needle sits at 9.8 of 16 — past the red mark at 8, and the flag does the arguing

02THE ARITHMETIC

two lines move the needle: what the crate weighs, what the context costs

the whole model, four lines

VRAM ≈ weights + KV cache + overhead
weights  = params × bits-per-weight / 8
KV cache = 2 × layers × kv_dim × context × 2 bytes   (f16)
overhead ≈ 0.7 GB CUDA/runtime + ~5% compute buffers

bits-per-weight from llama.cpp's ggml quant formats — the same numbers the files on Hugging Face are built with

Quant	bpw	Quant	bpw
Q2_K	3.35	Q5_K_M	5.69
Q3_K_M	3.91	Q6_K	6.59
IQ4_XS	4.25	Q8_0	8.50
Q4_K_M	4.85	F16	16.0

GQA—a Llama-3-class 8B keeps 8 KV heads × 128 dims × 32 layers, which is 128 KB per token

no GQA—a Llama 2 13B burns 640 KB per token — five times more, from an older and nominally smaller-era model

the flip—press key 3 on the weighbridge — an old 13B out-weighs a modern 70B's cache bill per token

03THE REVERSE LANE

budget in, quant out — a fixed card, a model in mind: the question people actually have

the record's own run: a 7B at 4,096 context against an 8 GB budget (GQA geometry)

Quant	Weights	Total (est.)	Verdict
Q4_K_M	4.24 GB	~5.7 GB	fits with headroom
Q6_K	5.77 GB	~7.3 GB	tight
Q8_0	7.44 GB	~9.0 GB	partial CPU offload, much slower

The file size told you what you would download. It never told you what you could run.

one screen, and the age-old "Q4 or Q8" debate collapses into your hardware's answer instead of somebody else's

04RUN IT YOURSELF

three inputs you can re-derive by hand, one verdict you can argue with — the defaults are the record's worked example

parameters

quant

context window

1k32,768 tokens128k

attention geometry

weights — 8 × 4.85 / 84.85 GB

KV cache — 128 KB × 32,7684.00 GB

overhead — 0.7 GB CUDA + 5% buffers0.94 GB

WEIGHS IN AT~9.8 GB

the "8 GB card" verdict: missed by 23% — the quant was never the villain; the context was.

card strip: solid · fits — dark · tight — past the dashed 85% line — hatched · over · partial offload

MoE routing—only the active experts get touched at inference, but this arithmetic prices the whole weight set, so MoE totals read high

mixed quant—Q4_K_M is itself an average across tensors — a real file lands a few percent off the table value

CUDA padding—the flat 0.7 GB varies by backend version — a middle-of-the-road figure, not a constant of nature

the header—when a decision is worth more than a few hundred megabytes, read the file's header with llama.cpp's gguf_dump.py

■ honest stamp Tested on: llama.cpp bpw tables · Llama-2/Llama-3 GQA geometry · in-browser, nothing uploaded. Recorded 2026-10-02, refreshed 2026-10-06; the tool lives as one HTML file, no dependencies, in github.com/mrsaynothing/gguf-vram-calculator. The weighbridge above runs the same arithmetic as the tool — the defaults reproduce the record's worked example (8B · Q4_K_M · 32k → ~9.8 GB, 23% over an 8 GB card) to the decimal.

end of the weighbridge · nothing fetched · everything local