■ space · prose lane

THE BRACKET BAND

Read your VRAM — nvidia-smi --query-gpu=memory.total --format=csv,noheader — then stand on your rung:

two rules make the band work: 1–2 GB headroom for the KV cache · quant down before size down

Two rules make this table work in practice: leave 1–2 GB of VRAM headroom for the KV cache — a 14B at Q4 plus an 8K-token context will not fit in a 10 GB budget, because context counts toward the total. And go one quant down before you go a model size down — Q5/Q4 quants cost a few percent of quality; a 7B instead of a 14B costs far more than that.

HOW MUCH VRAM DO YOU NEED?

The honest rule of thumb: VRAM needed ≈ quantised weights + 0.125 GB per 1K tokens of context at 8-bit KV cache. An 8K context is about 1 GB — so a 14B at Q4 (~9 GB) plus 8K tokens will not fit in a 10 GB budget. In plain numbers: 8 GB runs 7B–8B models at Q4 comfortably — autocomplete-length answers, 4K–8K context. 12 GB is the sweet spot for 14B at Q4 with a realistic 8K–16K coding context. 16 GB opens 20B-class dense models and Qwen3 14B with longer context. 24 GB runs the Qwen3 Coder 30B A3B MoE at Q4 — the closest thing to a cloud-quality coding model you can host yourself. CPU-only? It works — llama.cpp will happily run a 7B Q4 on a laptop CPU at 5–10 tok/s — but treat it as a patience exercise, not a daily driver.

QWEN3 CODER VS DEEPSEEK

The matchup the autocomplete bars actually ask about, and the answer splits cleanly:

• Qwen3 Coder (30B A3B) is built for the edit loop: it follows instruction-format conventions for tool calling, produces consistent diffs, and — the decisive part — the MoE design means only ~3B parameters activate per token, so a single 24 GB card gets 40–60 tok/s. For IDE-style use, responsiveness is quality.
• DeepSeek V3/R1 line argues at a higher level: architecture decisions, tricky algorithms, multi-step reasoning. But the flagship is 600B+ parameters; locally you only run it heavily quantised on multi-GPU or Mac unified-memory rigs, and it writes prose about code faster than it writes code.

Local choice: Qwen3 Coder for the daily driver, DeepSeek only if you have the hardware to host it near-full-precision. At 8–16 GB the debate is moot — Qwen2.5/3 Coder models are the strongest thing that fits.

FIVE COMMANDS TO WORKING

# 1. install Ollama (Linux)
curl -fsSL https://ollama.com/install.sh | sh

# 2. pull the model that fits your VRAM bracket (8 GB card shown)
ollama pull qwen2.5-coder:7b

# 3. chat with it interactively
ollama run qwen2.5-coder:7b

# 4. use it as an OpenAI-compatible API from any tool
curl http://localhost:11434/v1/chat/completions \
  -d '{
    "model": "qwen2.5-coder:7b",
    "messages": [{"role": "user", "content": "Refactor this fn to be async: add(a,b){return a+b}"}]
  }'

# 5. point tools that expect OPENAI_BASE_URL at it
export OPENAI_BASE_URL=http://localhost:11434/v1

verify: curl -s $OPENAI_BASE_URL/models → "qwen2.5-coder:7b" in the list

Where local wins, precisely: the edit loop — refactors, boilerplate, test scaffolding, "make this function async" — where latency is the product and the diff is small. Where it loses: long multi-file reasoning across an unfamiliar architecture, where frontier hosted models still earn their tokens. Run both against the same endpoint and use each where it wins.

end of the brackets · nothing fetched · everything local