How it works¶

One read, one distribution¶

polyjev answers each question with one or more reads. A read shows the model the state and one question whose options carry short labels, and returns a probability for every option:

State:
{"ticket": "Everything is down and we have a demo at noon."}

Question team: Which team owns this?
  A: billing
  B: outage (service down)
  C: feature

Reply with only the label of your answer (A, B, C) and nothing else.

Labels are yes / no for noul, A, B, … for choice, and 1 … 9 for score (letters past nine levels). The label set is the whole answer space, so no answer can fall outside your options.

Read strategies¶

How a read turns into probabilities depends on what the model exposes. strategy: auto (the default) picks the best one the model supports and remembers the choice.

strategy	used for	how
`logits`	in-process Hugging Face models	the exact next-token probabilities of the label tokens, from the full vocabulary
`logprobs`	vLLM, SGLang, llama.cpp, Ollama, OpenAI models that return logprobs, some Gemini models	one short completion with `top_logprobs=20`; label tokens' probabilities at the answer position, merged across variants (`"B"`, `" B"`, `"**B"`) and renormalised
`verbalized`	Claude, OpenAI reasoning models (GPT-5.x), anything without logprobs	structured output holding a probability per label (JSON Schema), normalised
`vote`	opt-in	sample an answer per read; the vote shares are the distribution

auto probes an unknown model once: a tiny request asking for logprobs. The answer is cached per endpoint and model (and on disk with cache_dir). If a model stops returning logprobs, or a thinking model puts no label in its first tokens, the engine falls back to verbalized for that question and records fallback in the diagnostics.

Which is better?

logits and logprobs read the model's actual belief and cost one short completion. verbalized asks the model to state a belief. That is usually looser, so calibrate it (see Calibration).

Samples and position bias¶

Models prefer some positions and labels (often whichever option comes first). Each extra read therefore shows a different layout:

choice: a cyclic shift of the options, so each option gets a different position and a different letter
noul: yes/no, then no/yes
score: ascending, then descending (labels stay tied to their level)

The reads are averaged. samples follows Jev and djev:

samples: N makes N reads. logprobs and logits stop at the number of distinct layouts, since a repeat would return the same numbers.
samples: "auto" (the default) reads once. If that read's entropy is above auto_threshold (0.1 nats), it makes up to auto_max (4) reads in total. So confident questions cost one read and uncertain ones get debiased.
vote starts with 3 reads and extends to max(auto_max, 6) when they disagree.

With several reads, each answer also reports stderr (the standard error of the top option's probability) and agreement (the share of reads that agree with the final answer).

Answers¶

noul: p (probability of yes), value (p >= 0.5), confidence
choice: value (your option, or an Enum member), name, probabilities, confidence
score: expected (the probability-weighted 0-based level, Jev's score), level / index / value of the most likely level, probabilities, confidence

confidence is the probability of the chosen option.

Extracting values: `span` and `spans`¶

Span("the invoice id") extracts one value from the state; Spans("every date") extracts all of them, in order. Answers are grounded by construction:

The model is asked to copy the value exactly.
polyjev locates the reply in the state's text, exactly first, then ignoring case and whitespace. When a chatty model wraps the value (Invoice id: A-1042, The id is "A-1042".), the value inside the reply is tried too.
The answer is the located substring with start/end character offsets, never the model's own spelling. A reply that is not in the state becomes found: false (see diagnostics.questions[id].note), so an answer can never contain text the state does not.

Offsets index the state when it is a string, its "text" field when it has one, and otherwise the JSON the model reads. Images can inform the answer, but a value that appears only in an image (not in the state's text) cannot be located, so it comes back as not found; ask a Choice or Noul about images instead.

Confidence is the probability of the weakest generated token along the value, including the token that ends it (the djev definition), when the model exposes logprobs. Otherwise it is the confidence the model states in structured output. Span questions cannot take part in depends_on / ask_if.

d = judge.decide(
    "Invoice #A-1042 from Northwind Traders. Total due: $1,234.56 by 2024-03-15.",
    {"invoice": pj.Span("the invoice id"), "amount": pj.Span("the total amount due"),
     "dates": pj.Spans("every date"), "po": pj.Span("the purchase order number")},
)
d["amount"].text, d["amount"].start, d["amount"].end   # '$1,234.56', 51, 60
d["po"].found                                           # False: there is none

Dependencies¶

depends_on: [ids] answers a question after those, with their answers in its prompt ("Answers so far: urgent: yes").
ask_if: {id: [answers]} asks the question only when that question's answer is among the listed ones. Otherwise the answer is None / null and diagnostics.skipped says why.
sequential: true answers questions one at a time, each seeing every earlier answer.

Questions without dependencies run concurrently.

Thinking first¶

think: N lets the model write a short note (up to N tokens) about each question before reading. The options are listed without labels in the note's prompt. Every read then gets the note plus its own labelled layout, so the note is shared and the debiasing still works.

Caching and cost¶

Every read of every question in a decision shares one prompt prefix: system text, images and state. Only the question block at the end differs, so:

vLLM's prefix cache reuses the state across questions and reads
OpenAI and Gemini apply their automatic prompt caching
Claude gets an explicit cache_control breakpoint on the state

d.usage counts input and output tokens across all reads, and d.diagnostics["cached_tokens"] shows how much came from cache.

Diagnostics¶

d.diagnostics (and the server response's diagnostics) reports:

the strategy, stages, skipped questions and sampling policy
timing
per question: reads, the entropy of each read, label_mass (how much probability the model put on valid labels), invalid reads, fallbacks and thoughts

A low label_mass means the model wanted to answer something other than your options.