Honesty line up front: this wave is days old — I've read the record, not run it. Sources: the Ollama announcement, llama.cpp 0.6.0 notes, checkpoint cards.

The bench with real latencies is the follow-up.

■ the answer first

Routing, gating, classifying → a decision model. A tiny model (144M–27B) returns {label, probability} in one pass — no prose to parse.

Both engines went local within a week: Ollama on September 29, llama.cpp's /v1/systemone in 0.6.0 on October 5.

Check the license before commercial use — OpenJev is CC BY-NC; the other five ship Apache-2.0.

WORDS FOR THE GALLERY

decision model — a tiny AI that answers a question with a pick and a confidence number, not a sentence
probability — the confidence number: 0.87 means "87 in 100 times I'd say the same"
routing — sending each incoming request to the right queue or worker, like a postie choosing a van
calibration — whether the confidence number tells the truth; a calibrated 0.9 is right 9 times in 10
GGUF — the file format local models ship in; the name in every download list

THE TWO SHAPES OF ANSWER

Chat models answer everything with prose — wrong shape for half of what pipelines need. A router wants which queue; a gate wants allow or deny, how sure.

A classifier wants one of six labels.

Ask a chat model for a decision: a sentence that means the right thing — usually — in different words every time, plus a parsing step that fails quietly.

One person sorts the post and writes an essay per letter. The other hands you a tray: "87% sure — bills." Who do you trust at 6am?

THE WRONG SHAPE · ask a chat model

30B chat modelwrites a sentenceregex / parsemaybe ≈0.9?

THE RIGHT SHAPE · ask a decision model

144M–27Bone pass{ "route": "billing", p: 0.87 }
{ "route": "sales", p: 0.09 }
{ "route": "other", p: 0.04 }same shape every call

THE VERDICT BANDS

The quiet feature: the probability is the answer. 0.51 and 0.99 are different decisions — set the line where acting beats escalating, and the tray sorts itself.

the act-above line

escalate belowact above 0.70act at or above

ticket · "refund the double charge"billing 0.87ACT

ticket · "charge?? (no text)"billing 0.51ESCALATE

gate · allow-or-deny checkdeny 0.93ACT

a chat model that says "this is probably billing" hides its own calibration — one phrasing, once. a decision model hands you the number; whether it tells the truth on YOUR labels is not in the release notes. that is what the bench is for.

WHAT THE WAVE ACTUALLY IS

"Jev" is the API contract: question in, fixed option list, probability per option. TypeSafe defined it; the local tools adopted it within a week.

Ollama shipped September 29 — v0.35 gained /v1/systemone, Nimble and Tev1 pullable at launch.

llama.cpp followed in 0.6.0 (October 5): the same endpoint, five open GGUFs — Julia-1, Laya, Kev-4B, lev, OpenJev — Nimble days later.

Tiny: 144M to 27B parameters; chat models start 30 times that. Small is the point — a decision is one pass, and instant changes which jobs ask.

One public accuracy comparison exists — Bespoke Labs' 13 datasets, 3,880 human-labeled decisions: Nimble 75.7%, Tev1-4B 73.3%, Tev1-0.8B 63.5%, hosted Jev 1.13 at 76.0%.

Vendor-adjacent numbers — a starting point. The bench that matters runs on YOUR labels.

picture it: a chat model is a senior consultant you book by the hour. a decision model is the rubber stamp on the front desk — it only says which tray, and it never gets tired. the size is the salary: the 144M does the job for pocket change; you pay the 27B only when the job needs to look at pictures.

THE MODELS ON THE SHELF

The llama.cpp shelf — the six checkpoints its 0.6.0 endpoint serves. Ollama's library carries its own set: Nimble, Tev1, Laya, Clef.

Filter by licence: OpenJev is the NC entry. MIT runtime doesn't change what the weights allow.

Checkpoint	Size	License	Notes
Julia-1	144M	Apache-2.0	2–20 options, 8,192-token combined limit
Laya	421M	Apache-2.0	English routing; don't assume multilingual
lev	4B	Apache-2.0	Text decisions
Kev-4B	4B	Apache-2.0	Reference-date preprocessing not bundled
Nimble	9B	Apache-2.0	LoRA on Qwen3.5-9B; text-only locally
OpenJev	27B	CC BY-NC 4.0	Vision input via its projector; non-commercial

A seventh shape, Clef, shipped in the same 0.6.0 release (Apache-2.0, text and vision); Cloudflare's Clef line pulls from Ollama too. Same idea, more doors.

The size ladder: 144M–421M on idle CPU for clean routing; 4B for subtle labels; 27B adds eyes. Run nvidia-smi --query-gpu=memory.total --format=csv,noheader — any card qualifies.

WHERE IT FITS — AND WHERE IT DOESN'T

It fits when the output is a decision: request routing (ticket → billing / sales / abuse), gates (allow-or-deny, with confidence), document triage at volume.

Classify with the small model; spend big-model tokens only on what it flags.

It doesn't fit when you need reasons. A label with a probability has no reasons attached. Need "why"? Back to a chat model — or the documents.

It doesn't fit when you need truth by consensus. Agreement is not verification — a unanimous wrong answer arrives in the most confident voice of all.

That's the failure our ledgers exist to catch.

verify the shape on your box: ollama run qwen3:0.6b "Reply YES or NO: is 7 prime?" — a one-token answer with no essay attached means the shape works.

■ honest stamp I've read the record, not run it. Runs on: ollama or llama.cpp — CPU fine at the small end. Recorded 2026-10-07 from the Ollama announcement, the llama.cpp 0.6.0 release notes and the published checkpoint cards; the accuracy line is Bespoke Labs' published 13-dataset run (3,880 human-labeled decisions), vendor-adjacent and quoted as found. The latencies are deliberately absent until they come off my own bench.

end of the courtroom · nothing fetched · everything local