A New Medical Benchmark Asks Whether AI Knows When It Doesn't Know
RadLE 2.0 scores AI models on radiology cases by punishing confident wrong answers and forgiving an honest 'I don't know'. Several models still hand out mistaken diagnoses with high confidence.
Most AI benchmarks ask one question: how many answers did the model get right? A new radiology benchmark from CRASH Lab at Ashoka University asks a second one that matters more in a hospital: when the model was wrong, did it at least sound unsure? Radiology’s Last Exam 2.0, released this month, is built around the idea that a confident wrong diagnosis is more dangerous than an honest admission of ignorance.
The setup is straightforward. Human radiologists and 16 AI models each work through 200 single-image “spot diagnosis” radiology cases. For every case they may give a diagnosis together with a confidence score from 0 to 4, or simply answer “I don’t know”. The main score, the Confidence Weighted Index, awards points for correct answers weighted by confidence and subtracts points for wrong ones the same way, while “I don’t know” earns neither credit nor penalty. A model that guesses loudly and often falls in the ranking even if its raw accuracy looks respectable. Four secondary scores round it out, covering reliability (how often confident answers are actually right), plain accuracy, safety (avoiding confident errors) and handover readiness (knowing when to pass the case to a human).
The findings are sobering without being dismissive. Several proprietary frontier models produced substantial numbers of high-confidence errors, meaning their confidence cannot be read as a signal that the diagnosis is trustworthy. Open-weight and medical vision-language models fared worse: some attempted nearly every case, produced few correct diagnoses, and delivered many of the wrong ones with moderate or high confidence. Human experts struck a better balance, though the researchers are explicit that radiologists are not perfect either. For context, in RadLE 1.0 last September expert radiologists scored around 83 percent while the best AI model reached about 30 percent, and within three months Gemini 3.0 Pro had overtaken radiology trainees. Capability is clearly moving fast. Calibration is not moving with it.
The reason this gap exists is almost mundane. If benchmarks reward accuracy alone, guessing is always the better strategy, so that is the behaviour training encourages. A paper in Nature by Kalai and colleagues made exactly this argument, and RadLE 2.0 is one attempt to change the incentive. It matters right now because patients are already uploading scans and records into general chatbots, and because grounding a model in real medical guidelines only works if the model first recognises that it does not know the answer.
What this means for you: If you have ever pasted a symptom or a scan into a chatbot, take this as the practical warning: a fluent, confident answer is not a second opinion, and the model has no reliable way to tell you how sure it should be. If you build or buy AI systems, the transferable lesson goes well beyond medicine. Ask any vendor not just how often the system is right, but what it does when it is uncertain, and whether it can hand a case back. A system that cannot say “I don’t know” is a system whose confident answers you cannot use.
Sources
SAP Just Spent Over a Billion Euros on AI That Reads Spreadsheets, Not Chats
SAP has closed its acquisition of Freiburg startup Prior Labs and committed more than one billion euros to it. The bet is on tabular foundation models, AI built for tables and databases rather than text.
