Deference Under Uncertainty: When Models Should Say I Don't Know
Abstract
Models are confidently wrong far more often than they admit. We study a training objective that rewards appropriate deference — acknowledging uncertainty when evidence is weak — and find it reduces high-confidence errors substantially while barely affecting usefulness on tasks the model can solve. The result suggests that honesty is not a trade-off against capability so much as a different allocation of confidence.
Most users will never read a log-likelihood. They interact with a model through its confidence: how certain it sounds. And certainty is where models most reliably mislead. A model that guesses and sounds sure is worse than a model that guesses and whispers.
This paper is about training that whispers. We call the target behavior appropriate deference — acknowledging when the evidence is weak, and asking or saying so instead of bluffing. We wanted to know two things: can we train for it directly, and does doing so cost usefulness where it should not?
Approach
We trained a variant of Mantus with a modified objective: alongside the standard next-token loss, we added a small loss term on calibration — rewarding the model when its expressed confidence tracked its true accuracy. We used a temperature-scaled, entropy-based confidence signal and evaluated the variant against the standard model on a held-out set of questions spanning factual recall, math, and ambiguous judgment tasks.
The key metric was not accuracy but honesty-weighted accuracy: correct, appropriately-confident answers counted positive; confident-but-wrong answers counted negative.
Results
The deferral-trained variant was less confidently wrong. Confident-but-wrong answers fell by 41% across the task set, and the biggest improvement was on ambiguous questions — the ones where a confident guess is most harmful. On solvable tasks, useful correct answers dropped by less than 2%, and the model’s calibration curve flattened toward the diagonal.
The model did not merely become more conservative. It retained its willingness to answer: abstention rates rose modestly (from 9% to 14%), but the abstentions clustered exactly where the standard model was wrong anyway. The net effect on honesty-weighted accuracy was a gain of 12 percentage points.
Discussion
Honesty, in this framing, is not the opposite of capability. It is a reallocation of confidence from places the model is wrong to places it is right. That is a friendlier trade than the field often assumes.
We are honest about limits: the confidence signal is a proxy, behavior on ambiguous questions is hard to benchmark objectively, and training for deference can drift into unhelpful hedging if pushed too far. But the direction is clear, and it is part of how Ichnus gets its “I don’t know.”
References
- Costa, R., & Banerjee, A. (2024). Calibration as a training signal. ICLR 2024.
- Kadavy, P., et al. (2025). When models hedge: a taxonomy of uncertainty language. ACL 2025.
- Lindgren, M., & Raman, P. (2025). Deference and its discontents. Curos Internal Report 2025-08.
- Nguyen, V. (2023). Confidence and consequence in LLM answers. NeurIPS 2023.
- Schwartz, D., et al. (2024). Abstention as a policy choice. ICML 2024.
- Yilmaz, S., et al. (2025). Measuring overconfidence in deployed assistants. FAccT 2025.