← Back to the site
Research
Retrieval that admits doubt
A study of when a retrieval-augmented model should answer "I'm not sure", with a benchmark of 3,000 hard questions.
3,000
hand-checked questions
−38%
confidently wrong answers
2025
workshop paper
Notes
The question
Retrieval helps a model answer from sources, but it also makes wrong answers sound sure of themselves. We wanted to know when the model should hold back.
What we did
We wrote 3,000 questions where the sources disagree, are out of date or are missing, and checked every answer by hand.
What we found
A simple agreement score across retrieved passages cut confidently wrong answers by 38% while keeping most correct ones. The benchmark and code are public.