← Glossary Term
AA-Omniscience
A benchmark that rewards a model for saying 'I do not know' instead of inventing an answer.
Most benchmarks only ask whether an answer was right. AA-Omniscience, run by the analysis firm Artificial Analysis, also asks what a model does when it has no idea. Confidently wrong answers are penalised, and honestly declining is credited, so a model can improve its score without learning a single new fact simply by guessing less.
This makes it one of the more useful numbers to check if you rely on AI for research or anything factual. A model can be knowledgeable and still be a bad research assistant if it fills every gap with something plausible. Hallucination rates from this benchmark tell you which failure mode you are buying.
Mentioned in