Decision Models: Jev, Clef, and Answers Without Generation

A router that asks an LLM "which of these four teams should take this ticket?" pays for a prompt, a reasoning trace, and a generated label it then has to parse. A decision model reads the prompt once and returns a probability for each team, without writing a token. TypeSafe, Cloudflare, Perplexity and OpenAI, among others, shipped one between 15 September and 22 October 2026.

The problem: decisions inside pipelines

Most model calls inside agent loops and pipelines are decisions with a known answer set: route a ticket, triage an alert, block or allow a message, pick a tool, grade an answer against a rubric. An LLM makes them by generating text. That costs a reasoning trace you didn't need, a label you must parse, failures when the output isn't one of the labels, and no probability you can put a threshold on. Structured output fixes the parsing; it doesn't fix the cost, and it still gives you a sample, not a distribution.

Jev and the typed-question API

The API is the idea. You send a state (text or JSON) and a set of typed questions; you get typed answers with probabilities. Jev, TypeSafe's model (TypeSafe calls the category "System One", after Kahneman's fast system), launched in early access on 15 September 2026 with four question types, which TypeSafe calls primitives:

Every question is evaluated on its own against the same state. The format caught on: SGLang, vLLM and Ollama all serve an endpoint called /v1/systemone that takes it. Here is a request in that format to Cloudflare's open decision model, Clef-Flash, served by vLLM (nightly 0.32.1rc1.dev564+g78e2f7960 of 26 October 2026, one GB10, BF16), about a support ticket asking for a duplicate charge to be refunded. The request and answers are trimmed to two of its four questions, the ticket text is shortened (and the ticket-id line our client added is dropped), the response envelope and diagnostics are omitted, and probabilities are rounded; usage is the full request's:

{
  "state": "Subject: charged twice\nHi, I ordered a pair of running shoes (order #48213) on Monday and my card was charged twice, 129.90 EUR each time. [...] Can you refund the duplicate charge? I need the money back before the end of the month. Thanks, Claire",
  "questions": {
    "team":   {"type": "choice", "instructions": "Which team should handle this ticket?",
               "criteria": {"billing": "payments, charges and refunds", "shipping": "deliveries and carriers",
                            "security": "account access and fraud", "product": "questions about items"}},
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?", "criteria": null}
  }
}

{
  "answers": {
    "team":   {"type": "choice", "choice": "billing",
               "probabilities": {"billing": 0.981, "shipping": 0.003, "security": 0.014, "product": 0.002},
               "confidence": 0.981},
    "refund": {"type": "noul", "noul": 0.664, "probabilities": {"yes": 0.664, "no": 0.336},
               "confidence": 0.664}
  },
  "usage": {"input_tokens": 528, "output_tokens": 0}
}

output_tokens is 0: nothing was generated. The second answer shows what you now own: the threshold. The customer plainly asks for a refund, and Clef-Flash gives yes 0.66. A threshold at 0.8 would send this ticket down the wrong path. Tune thresholds on your own labelled traffic.

Jev costs $0.042 per million input tokens, and output is free (models page). A request holds 64K tokens and a question 32K; a header raises both to 262K, at some cost in speed and accuracy by TypeSafe's account. Production input is text. TypeSafe gives an end-to-end response time of 70–500 ms, and "193.6x faster, 444.6x cheaper" on four of its workflow evals, scored against the average of GPT-6 Astra and Fable 5.1 as the reference answer, figures it calls "on the higher end" and whose evals were generally run from laptops on the West Coast. Its architecture is not disclosed. TypeSafe does publish where Jev fails, which most vendors don't: literal reading, counting and maths, date comparisons, indirection, distracting state, instructions that contradict their criteria, adversarial content, and a bias toward the first option of a Choice, among others (jaggedness page).

How it differs from an LLM

A decision model is an LLM's prefill with a different ending. There is no decode loop: the prompt goes through the network once, and a small output layer turns the final hidden states into a probability for each declared answer. Latency is set by the prompt length, the price is on input only, and calibration, how well a 0.9 means "right nine times in ten", is a training objective rather than an accident.

A decision model's cost is its prompt: no decode loop, and output is free at every vendor that sells one.

You lose what decoding gives you: reasoning steps, explanations, open-ended answers, multi-step work. A decision model can't think its way through a question it gets wrong at first sight, and it has no reasoning setting to turn up.

The models in this piece train an output layer on an open backbone: Clef, Perplexity's Decider and vLLM Semantic Router's d3 all start from a Qwen model and read its hidden states. There is also a cheap substitute that runs on any model: number or letter the options, ask for one label, and read the probability of each label token at the first position of the reply. vLLM and SGLang do this for generation models behind the same /v1/systemone endpoint. It is not the same thing: each question is its own read of the whole prompt, the probabilities were never trained for calibration, and some always land on tokens that aren't labels (vLLM reports what's left on the labels as label_mass). It is still the baseline to beat, as the serving section shows.

One design choice splits the trained models. Decider and d3 make some attention layers noncausal: the state's representation depends on the questions that follow it, so nothing about the state can be computed once and reused. Clef stays causal, but its head reads the last hidden state of every prompt token, which a prefix cache doesn't keep, so in practice it runs as one full prefill per request too.

Inside one: Clef

Clef is the clearest example because Cloudflare released everything: weights under Apache-2.0, the configuration, and the reference code that turns a request into probabilities. It is a Qwen backbone with a small trained head:

According to Cloudflare, the head was trained jointly with rank-256 LoRA adapters on the frozen backbone, with a label-smoothed cross-entropy plus a Brier loss on synthetic data, and a secondary calibration objective it names RLCD, the name TypeSafe gave its own method two weeks earlier. The released checkpoints are full weights, with the adapters merged.

The reference implementation, joint_schema_model.py, shows the whole mechanism:

1. One prompt for the whole request. A fixed system line ("Decide every field jointly"), then STATE: and the state (JSON compact, keys sorted), then SCHEMA FIELDS: with each question's id, type, instruction and allowed options, and finally an empty thinking block and JOINT SCHEMA DECISIONS:. Each piece is tokenized separately, so the code knows exactly which tokens belong to each question and each option.
2. Options sorted by name. A Choice's options are reordered alphabetically before they reach the model, so the order you send them in can't bias the answer, unlike letter-label reads, where first-option bias is on TypeSafe's own list.
3. One prefill, no cache. The backbone runs once over the prompt with use_cache=False: nothing is decoded, so there is no KV cache to keep.
4. A head over spans. The head averages the backbone's last hidden states over each question's tokens and each option's tokens, lets the option vectors attend to the whole prompt (two "evidence routing" layers), lets the questions attend to each other (four layers), and gives each option one logit: a lexical prior, comparing the option's output embeddings with the question, plus a gated learned score. A softmax per question turns logits into probabilities.

Every option of every question is scored from the same forward pass: ten questions cost one prefill of a longer prompt (the schema lists every option), not ten prefills.

Clef decides all the questions of a request jointly, so an answer can depend on the other questions, where Jev evaluates each in isolation. Cloudflare's hosted Clef-Flash accepts 24K tokens and Clef 64K; the reference code defaults to 16,384 (see Gotchas).

The others, and how to compare them

Within six weeks there were a dozen decision models. The best known, with the only shared yardstick, the community Decision Index (edition 0.4.1, board as of 26 October 2026; other entrants sit between some of these rows):

OpenAI's entry is an API rather than a model you can download: POST /v1/decisions on gpt-6-luna, in beta from 6 October and generally available from 22 October, with predicate, choice and score questions, up to 400 per request, output free, and a claim of "about 10x faster than the Responses API" without a stated setup.

The index's headline score is a held-out score: 60% from private sets that test the same skills as the public benchmarks, 40% from private sets in new domains. It also publishes a raw score on the public sets, and the difference matters. Clef beats Jev on the public sets (71.07 against 69.98), which is what Cloudflare's own run of the suite (version 0.2.1) measured, and trails it on the held-out score (54.92 against 60.24). When a vendor says its model leads, check which sets it means. Two rules are worth knowing: unanswered counts as wrong, and entrants may train on benchmarks' official training splits, which Perplexity says its v1.2 does.

What the evidence says

Independent studies agree that decision models are cheap and reasonably calibrated, and that their mistakes are the ones an LLM judge would make anyway. On injection they disagree about how often, not about whether.

Calibration. A study from the University of Bonn and Fraunhofer IAIS (arXiv 2609.37647) ran Jev 1.13 on 37 datasets, 346,009 requests, for $9.40 in total. Its mean calibration error (ECE) over 33 datasets was 0.074, the same as Qwen3.8-27B read through its next-token probabilities (0.075), and much better than Gemma-4-E4B (0.184). The Decision Index measures calibration too, and there Clef (0.040) and Clef-Flash (0.025) beat Jev (0.080).

Judging. Rao and Callison-Burch compared Jev and Decider v1.1 with three flash-tier LLM judges on nine rubric panels with human labels. The LLM judges cost 16 to 357 times more and took 5 to 350 times longer; the decision models trailed significantly in at most 6 of 27 comparisons. The finding that changes how you build a pipeline: more than 85% of the LLM judges' verdicts repeated the decision models' most confident errors. Sending low-confidence cases to an LLM gained at most 2.7 points, even with thresholds chosen in hindsight. A cascade saves money; it doesn't buy much accuracy.

Injection. Check Point attacked a due-diligence assistant built on Jev with three adaptive attacker agents over ten turns, the attacker seeing the probabilities. Jev's verdict was flipped in 59% of attempts, at $0.54 per success, by planted fabricated evidence rather than instructions; marking the document as untrusted made no difference. The LLMs in the same test fell to 100% and 67% without reasoning, and to 70% and 19% with it: reasoning was the best defence measured, and Jev has none to turn on. A study on 510 fixed InjecAgent cases (arXiv 2609.28613) found the attacker's target rarely chosen, with adaptive attacks raising success on fresh calls from 1.8% to 3.5%. The likely difference is the attacker and the method: fixed injected instructions rarely work, an adaptive attacker who plants evidence and sees the probabilities usually does. Either way, the typed output is not a security boundary.

Serving one yourself

Open decision models run on the usual engines today, but support depends on the engine, the release, and the endpoint (as of 26 October 2026):

On vLLM, Clef-Flash needs a build of main. This is what we ran, on the same machine:

$ uv pip install -U vllm --prerelease=allow \
    --extra-index-url https://wheels.vllm.ai/nightly \
    --index-strategy unsafe-best-match
$ vllm serve Cloudflare/clef-flash --max-model-len 16384

The nightly index installs whatever main is that day (we ran commit 78e2f79). --max-model-len 16384 matches the bound in Clef's reference code; longer states are cut silently (see Gotchas). The server then answers POST /v1/systemone on port 8000 with the request shown earlier.

On vLLM v0.32.0 the same vllm serve command starts without error and answers with HTTP 200, but it serves the bare Qwen3.5 backbone with letter reads, not Clef. Check the response: Clef returns output_tokens: 0 and label_mass: 1; the backbone returned output_tokens: 2 and a label_mass of 0.32 on a yes/no question. SGLang v0.5.22 serves Clef in embedding mode, with its radix cache, chunked prefill and CUDA graphs off (cookbook), and at tensor parallelism 1 because the head reads whole rows of the output embedding matrix (#42721). Neither engine serves Clef on /v1/decisions, and the two engines' /v1/decisions share a path, not a contract: vLLM's follows OpenAI's schema, SGLang's has its own.

Speed is not why you'd pick Clef-Flash. On the same machine, it answered 24 questions about a long ticket (about 2,200 tokens of state in Clef's tokenizer, 2,700 in Qwen3-8B's) in 0.94 s at the median, and Qwen3-8B on vLLM 0.32.0, answering by label reads with prefix caching, took 0.94 s too. What Clef-Flash adds is answers that always land on your options, a calibration it was trained for, and questions decided together. Decisions in vLLM has the measurements and both kinds of response side by side.

Gotchas

confidence is not portable. The same field name hides different formulas, so a threshold of 0.8 tuned on one provider routes differently on another. Threshold on the probabilities, which every provider returns, or re-tune per provider:

Long states are cut silently. Clef's reference code, vLLM and SGLang truncate the state to fit 16,384 tokens once the schema is placed, without an error. A decision about a long contract may be made on its first pages. Count tokens before you send, or split the document.

Label reads count yes in lowercase only. On vLLM, a generation model's yes/no read sums the probability of the lowercase yes and no tokens; a confident Yes lowers label_mass and the confidence with it.

Usage counts the state once per question on generation models. Each question is its own read of the whole prompt, so 24 questions about a 2,700-token state report about 65K input tokens, most of them prefix-cache hits. Budget on what was computed, not on that number.

Verdict

For decisions with a fixed answer set, at a volume where an LLM's latency or price hurts (routing, triage, guardrail pre-filters, rubric grading at scale), a decision model is the right tool today. It is the wrong one for open answers, multi-step reasoning, the cases on TypeSafe's own failure list, and anything an adversary can write into. On the held-out index, open models already lead: GLiDE 1.2, d3-ultra and pplx-decider v1.2 all score above Jev, the last two under Apache-2.0. Clef scores 15 points below d3-ultra but runs on every engine in the table, and its reference code is public. The d3 family runs on vLLM main and Decider on SGLang v0.5.22: if your engine has them, benchmark them on your traffic before settling on Clef.

What would change that call: a top-ranked open model with merged support in vLLM and SGLang, TypeSafe publishing Jev's architecture, or a training result that makes decision models resist fabricated evidence.

If you build agent pipelines, start with label reads on the model you already serve and watch label_mass; move to a trained decision model when label_mass drops or thresholds misroute. If you run a gateway, serve Clef on SGLang v0.5.22 or a vLLM build of main, never on v0.32.0, and try d3 or Decider where your engine has them. If the decision protects something, never make it on untrusted content alone.

Next: Decisions in vLLM serves both kinds side by side and measures what each costs.