Paper, researchers, and primary source
Major lab research / evaluation
The paper evaluates whether language models can estimate the probability that their own answers are correct, including via verbalized confidence and learned probes.
Contribution 1
Measured self-evaluation across knowledge and multiple-choice tasks.
Contribution 2
Compared direct probability predictions with verbal confidence formats.
Contribution 3
Studied calibration, task transfer, and the relationship between confidence and correctness.
Research context
evaluation / 2022
Language Models (Mostly) Know What They Know places calibration inside the broader evaluation discussion at Anthropic, with uncertainty supplying a second analytical lens. The editorial sequence connects three claims: Measured self-evaluation across knowledge and multiple-choice tasks; Compared direct probability predictions with verbal confidence formats; and Studied calibration, task transfer, and the relationship between confidence and correctness. The combination matters because self evaluation only has meaning under the paper's stated setup. Operationally, the record points to one consequence: model confidence can be informative, but it must be calibrated on the relevant model and task before being shown as reliability.
Methods and evidence reading
1 cataloged method notes
Method 1
The experimental design in Language Models (Mostly) Know What They Know tests measured self-evaluation across knowledge and multiple-choice tasks and compared direct probability predictions with verbal confidence formats against the paper's documented baselines, datasets, model variants, or systems workloads.
How to read the evidence
For Language Models (Mostly) Know What They Know, the useful question is what the documented setup supports and where transfer remains untested. Its evidence chain begins with two documented moves: Measured self-evaluation across knowledge and multiple-choice tasks; and Compared direct probability predictions with verbal confidence formats. Reported evidence then addresses: Studied calibration, task transfer, and the relationship between confidence and correctness. The resulting interpretation is practical but conditional: model confidence can be informative, but it must be calibrated on the relevant model and task before being shown as reliability. Its boundary is that reading Language Models (Mostly) Know What They Know as evidence for calibration requires preserving selected threat model, evaluator models, prompt sampling, model revisions, construct validity, and rater instructions; changing those conditions creates a new experiment. Any extension should report how altered uncertainty conditions affect the original calibration result. An independent check of Language Models (Mostly) Know What They Know needs a fixed calibration comparison, a declared uncertainty variation, and saved cases where Compared direct probability predictions with verbal confidence formats does not carry over.
Findings in the source record
1 paper-specific findings
- The reported evidence in Language Models (Mostly) Know What They Know supports studied calibration, task transfer, and the relationship between confidence and correctness.
Practical implication for AI builders
Anthropic / 2022
Proposed BrokenGPT application
Research blueprint / proposed status
Proposed route placement / calibration
For a proposed BrokenGPT experiment based on Language Models (Mostly) Know What They Know, calibrate confidence per endpoint and task family, and use it to trigger retrieval or human escalation rather than presenting raw self-reports as certainty. Keep the calibration path isolated, versioned, and attributable to this research record.
Proposed acceptance test / uncertainty
Validate the proposed calibration route against the paper's reported outcome: Studied calibration, task transfer, and the relationship between confidence and correctness. Before a proposed Language Models (Mostly) Know What They Know change advances, compare calibration, slice stability, metric coverage, and evaluator agreement and inspect uncertainty counterexamples outside the aggregate calibration result.
Proposed decision boundary / self evaluation
Balance breadth, repeatability, and construct validity before promoting the proposed self evaluation design. Because reusing the mechanism calls for separate evidence about deployment drift, language coverage, unsampled behaviors, adversarial adaptation, and judge bias, not an inference from the original benchmark alone, adoption remains conditional on replay under BrokenGPT's selected model, runtime, and policy configuration.
Limitations, verification, and source
Boundaries recorded with the paper
Limitations
- Reading Language Models (Mostly) Know What They Know as evidence for calibration requires preserving selected threat model, evaluator models, prompt sampling, model revisions, construct validity, and rater instructions; changing those conditions creates a new experiment.
- Reusing the mechanism calls for separate evidence about deployment drift, language coverage, unsampled behaviors, adversarial adaptation, and judge bias, not an inference from the original benchmark alone.
PRIMARY SOURCES
- 01Language Models (Mostly) Know What They Know
Anthropic — Primary primary arXiv paper / 11 July 2022 / Saurav Kadavath, Tom Conerly, Amanda Askell, and 33 more