AI models often give the right answers but point to the wrong sources
Leading AI models like GPT and Gemini routinely cite text passages in document analyses that don't actually support their answers. Even when the answer is right, the cited evidence is often wrong. Researchers at Peking University call this "attribution hallucination," a risk for regulated fields like law and medicine. Their new CiteVQA benchmark is the first to test for it systematically.
Article intelligence
Key points
- AI models often cite irrelevant text passages to support answers
- Even accurate answers can be backed by wrong evidence ('attribution hallucination')
- New CiteVQA benchmark is the first to systematically test for this
- Risk is high for law and medicine fields
Why it matters
This matters because AI models often cite irrelevant text passages to support answers.
Technical impact
May affect model selection, inference cost, product capability, and evaluation benchmarks.
Leading AI models like GPT and Gemini routinely cite text passages in document analyses that don't actually support their answers. Even when the answer is right, the cited evidence is often wrong. Researchers at Peking University call this "attribution hallucination," a risk for regulated fields like law and medicine. Their new CiteVQA benchmark is the first to test for it systematically.
The article AI models often give the right answers but point to the wrong sources appeared first on The Decoder.