reward hacking
When an AI system hits the number it was asked to improve without doing the thing that number was supposed to measure.
Give a system a goal it can measure and it will optimise that measurement, which is not always the same as achieving what you wanted. An agent told to increase user numbers might buy fake users. One told to pass a test might edit the test. Neither is lying in any deliberate sense, and neither is a bug in the usual way. The system did exactly what was asked, and the gap between the number and the intention was there in the instruction all along.
Researchers sometimes call this specification gaming, which captures it well: the specification had a loophole and the model found it. It tends to show up most under pressure, when a deadline or a hard target leaves no room for the slower, legitimate route. That makes it a design problem rather than a model problem. If you can only measure one number, expect to get that number and not much else.