Researchers asked why chatbots are bad at spreadsheets, and the answer is stranger than expected
A new paper tests five explanations for why large language models lose to decades old methods on tabular prediction. In low dimensions the model behaves like a simple distance based classifier. In higher dimensions nothing classical explains it.
Language models write code, summarise contracts and pass exams. They are conspicuously bad at one of the most common jobs in all of machine learning: looking at a table of rows and columns and predicting the missing value. A paper published on 3 August by Marta Garnelo and Wojciech Czarnecki sets out to find out why, and the interesting part is how much of the obvious explanation it rules out.
The setup is deliberately bare. The researchers hand a frontier model a single prompt containing the entire training table and the test rows, and ask for predictions in one generation pass. No tools, no code execution, no fine tuning, no agent loop. Then they test five candidate explanations: that the model cannot cope with noisy or non linear data; that flattening a table into comma separated text destroys the column structure; that the way numbers get chopped into tokens mangles them; that asking for too many predictions per query degrades quality; and that the number of columns is the problem. They also compare the model’s behaviour against 252 configured classical models, the kind of decades old statistical methods that still win these tasks.
What is behind this. The finding that stands out is the mismatch between two regimes. With only two columns, the model behaves remarkably like a simple distance based method, the sort that predicts a point by looking at whatever is nearest to it, with up to 91.6 percent agreement on a prediction grid. Add more columns and that resemblance breaks: no classical model reproduces the predictions, even when the researchers deliberately add tuned noise to try to imitate them. In other words, the model is not doing a bad version of statistics. It is doing something else, and that something else does not scale. This matters beyond curiosity, because it is the founding premise of tabular foundation models, a whole young research field building models specifically for tables, and until now the “why” was mostly assumed.
What this means for you: the practical rule is simple and worth holding on to. If your question is “predict this number from this table”, a chatbot is the wrong tool, and a boring old method such as gradient boosting or even logistic regression will beat it, often by a lot. Where a language model does help with tabular data is around the edges: explaining what a column means, writing the code that runs the actual model, spotting formatting problems, describing results in plain language. This is a good example of a general habit worth building. The impressive breadth of these tools makes it easy to assume competence everywhere, and the honest answer is that the failures are patchy and rarely announce themselves.
Sources
Source: https://arxiv.org/abs/2608.02412
Mistral released a free safety filter that you describe in plain English, and it fits on one graphics card
Shieldstral is a 3 billion parameter open-weights model that checks text and images against a policy you write yourself, in ordinary sentences, without any retraining. It runs on a single 16 GB GPU under an Apache 2.0 licence.
