Alibaba's newest model scores higher and guesses more: hallucination rate jumps from 23 to 40 percent
Qwen3.8 Max gained 10 points on the Artificial Analysis Intelligence Index, matching Claude Opus 4.8. It also got slower, more expensive per task, and far more willing to invent an answer rather than admit it does not know.
Alibaba’s Qwen3.8 Max scores 56 on the Artificial Analysis Intelligence Index, a 10-point jump over its predecessor Qwen3.7 Max and enough to draw level with Claude Opus 4.8. On GDPval-AA, a benchmark built around realistic work tasks, it leapt 468 Elo points to 1,739, passing Kimi K3 and leaving only Claude Opus 5 ahead of it. That is a genuinely large step in one version.
Now the rest of the table. Kimi K3 still scores one point higher at 57, and does it for about 25 percent less money. And the way Qwen3.8 Max earns its score is expensive: it takes 64 steps per task where the previous version took 14, and the input text it processes grew fifteenfold, because each step resends the whole conversation so far. Alibaba cut its prices, with input dropping from 2.50 to 2.00 dollars per million tokens and output from 7.50 to 6.00, and the cost per task still more than doubled, from 0.53 to 1.14 dollars. Kimi K3 costs 0.86, GLM-5.2 costs 0.57.
Two scores also went backwards. AA-LCR, which tests whether a model can pull together facts scattered across a very long document, fell 2 points. AA-Omniscience fell 10. That second one is the interesting number, because of what it measures.
What is actually going on here
AA-Omniscience does not just ask whether a model answers correctly. It also rewards saying “I do not know”. Qwen3.8 Max got about 31 percent of answers right, roughly the same as before. But its hallucination rate, the share of answers that were confidently wrong rather than declined, climbed from 23 to 40 percent. In plain terms: it did not get more knowledgeable, it got bolder. It now guesses in situations where the previous version would have admitted a gap.
That tradeoff is easy to produce by accident. Training a model to be more helpful, more persistent and better at long multi-step tasks pushes it toward always producing something. The same instinct that makes it grind through 64 steps to finish a job also makes it fill in a fact it does not have. This is a good example of why one headline number never describes a model, and why the metric you should care about depends entirely on what you use it for.
What this means for you: if you are choosing a model, look past the top-line score to two things: cost per completed task, not price per million tokens, and how the model behaves when it does not know something. For research, summarising documents or anything you will act on, a model that admits gaps is worth more than one that scores two points higher. For long, structured work where you check the output anyway, the thorough-but-pricey profile may suit you fine. And whichever you use, treat a confident tone as a writing style, not evidence.
Sources
When AI writes a kids' story about animals, the female characters almost disappear
A University of Washington study ran nearly 24,000 story completions through six leading models. Two percent of the animal characters came out female, 41 percent male, and 57 percent neutral or ungendered. The researchers argue the guardrails meant to prevent bias caused it.
