Overconfidence as Self-Misrepresentation: Language Models Verbally Discard the Uncertainty Their Own Logits Encode
Abstract
A language model’s stated confidence is increasingly read as evidence about the model, yet rarely checked against what the model’s own com- putation encodes. Across two instruction-tuned models (GEMMA-3-27B-IT and QWEN3-32B), we find that both, when they state a confidence in words,4 discard the uncertainty their own logits represent: verbalized confidence sits near 0.95 regardless of correctness and barely ranks right answers above wrong (AUROC ≈0.57), while the top-two margin over the answer-option7 logits, read from the same generation, ranks them well (≈0.75). The gap is large, replicates across architectures, and is a property of verbalization rather than the softmax. It is localized: within the very answers a model declares itself at least 90% confident of, its logits still separate correct from incor- rect. And it is robust: neither steering the model’s representations—with a self-representation vector or any control direction—nor prompting it to be confident moves the verbalized overconfidence. We read this miscalibra- tion as self-misrepresentation—information a model represents but does not report—and, because it resists intervention, argue that the safe response for agent oversight is diagnostic: read what the model computes, not what it says.