How does an AI agent know when it is not sure?

Not from its own confidence. The signal with a published number behind it is disagreement: ask the same question ten times, measure whether the answers mean the same thing, and you separate the wrong answers from the right ones at 0.790 AUROC. Asking the model whether its own answer is true scores 0.698.

That second number is the one to sit with. A stated confidence comes out of the same sampling pass as the answer, so it carries the same error. It is a token, not a measurement.

Why can you not ask the model how confident it is?

Because the training that makes a model good at your task makes it worse at declining. AbstentionBench, by Kirichenko and colleagues, assembled 20 datasets of questions that have no answer: unknowns, underspecified prompts, false premises, outdated facts. Across 20 frontier models, reasoning fine-tuning reduced abstention by 24% on average, including in the maths and science domains those models are tuned for.

Two of their findings matter more than that headline. Scaling is of little use, so the next model does not fix this for you. And a carefully written system prompt does raise the abstention rate, while the underlying inability to reason about uncertainty stays where it was. You can prompt a model into saying it is unsure more often. You cannot prompt it into being right about when.

A model trained to answer answers.

Is there a confidence number in the API at all?

Not on the API we build on. The Messages API reference lists no logprobs or top_logprobs parameter and no token-probability field in the response, so the old approach of thresholding on the probability of the first token is not available to write.

It got narrower this month. API version 2026-10-21 removed temperature, top_p and top_k, and a request carrying any of them now returns a 400 on every model. The spread of your samples is no longer a parameter you set. You still get variation, because you always did: a thousand identical calls at temperature 0 came back as 80 distinct completions. What is gone is the dial, so you cannot widen that spread to make disagreement easier to read, or narrow it to make one answer reproducible. Checked by Unity Horizon on 27 October 2026.

What does a usable uncertainty signal cost?

Ten generations per answer, at the best published number. Farquhar and colleagues, published in Nature in 2024, sampled each prompt ten times, clustered the answers by meaning rather than by wording, and measured the entropy of those clusters.

Signal	AUROC	What it needs
Semantic entropy over ten samples	0.790	ten generations per answer
The model's own judgement of its answer	0.698	one extra call
Naive token entropy	0.691	token probabilities, which the Messages API does not return
Supervised embedding regression	0.687	labelled examples of your own failures

Averages over 30 task and model combinations. Read 0.790 like this: hand the signal one wrong answer and one right answer, and it calls the wrong one more uncertain 79% of the time. The paper also gives a discrete variant for models that expose no output probabilities, and reports it performing similarly, which is what anyone on a closed API is left with.

Ten samples is a tenfold bill on the generation, and the follow-up work treats that as the problem. Hallucination Detection on a Budget calls ten generations per prompt "prohibitively expensive in many practical cases" and reaches the same AUROC with 53% of the samples. At five samples its table puts Llama-2 on TriviaQA at 0.734 plus or minus 0.007.

Is sampling ten times per message too expensive?

Work it out before deciding. Claude Haiku 5.5 is $0.10 per million input tokens and $0.50 per million output for prompts up to 100,000 tokens, with cache hits at $0.01. Ten samples of a 300-token reply is 3,000 output tokens, or $0.0015. Ten cache reads of a 4,000-token prompt is 40,000 tokens, or $0.0004. Call the whole check a fifth of a cent per message, arithmetic over list prices on 27 October 2026.

Set that against the hour it protects. At the EUR 14 loaded hourly cost we use in our own filing estimates, one minute of a person is EUR 0.23, which buys about 120 of these checks. At that unit price the sampling is never the binding cost; the budget line is the person who reads what the agent escalates.

Which is the real constraint on the threshold. Sample ten times on every message and the bill barely moves. Send a tenth of your messages to a human and you have hired someone. The token price does not set that threshold. The staffing does.

So what should trigger the handoff?

Something the model did not say. The triggers that hold up in production are all observable before anyone trusts the answer, and we would wire these five first:

• retrieval came back under the similarity threshold, or came back empty
• a tool returned an error, a timeout, an empty body, or a field the schema did not expect
• a validation rule failed, the arithmetic kind rather than the judgement kind, like an ΑΦΜ that fails its checksum or a date outside the booking window
• the action cannot be undone: submit, refund, send, cancel
• the request falls into a class somebody decided about in advance, with a name and a date on the decision

Sampled disagreement is the sixth, and it goes after those, because it costs ten calls where the five above cost nothing. Same shape as putting a credential below the model rather than inside its context. The check that holds is the one the model cannot talk its way through.

We built askHermis, and askHermis answers 85% of guest queries instantly, so the design has always lived in the other 15%. We also built OFFSET, which files declarations directly to the ΑΑΔΕ portal, where a misread field costs an amended filing and an awkward phone call, and the shape that survives there is prepare, approve, submit. The model reads the message. The validator reads the consequence.

Who gains from a confidence score, and who pays?

Vendors who print a percentage beside each answer gain, because a confidence column demos beautifully and nobody in the room asks what it was scored against. Teams with a labelled set of their own past failures gain honestly: the fourth row of the table above is weak in general and can be the strongest signal you own on one narrow task, because you have the labels and the benchmark does not.

The bill lands on the team whose agent writes to a system where a wrong write is expensive, and who read the number in the dashboard as an instrument. It lands again on the operator who learns from a review, or from an amended declaration, that the agent was 94% sure.

What would we check first?

Count your escalations against your errors. Pull the cases the agent handed to a person last month, then pull the cases it answered wrong and nobody caught, and look at what the two sets have in common. If the second set is larger, the rule is catching the wrong cases.

Then pick the threshold from the staffing. One person can absorb 20 escalations a day and not 200, and that ceiling decides which signal you can afford long before any benchmark does.

What is not measured here should be said out loud. The 0.790 was scored on short-answer question sets like TriviaQA and SQuAD, not on an agent deciding whether to press submit on a form, and we found no published figure for the agent case. The cost lines above are arithmetic over list prices rather than a bill we have paid at that volume.

What does your agent do today when retrieval comes back empty, and who decided that? If the answer is that it answers anyway, that is where we would start.