Karpathy turned one paragraph of Tolkien into a 3D scene for ten dollars, and called it a vibe check

Andrej Karpathy handed Claude Opus 5 a single paragraph from the opening of The Lord of the Rings and asked for a 3D scene. Two hours and roughly one million tokens later, about ten dollars worth, the model produced 5,500 lines of code rendering the scene in Three.js, a JavaScript library for 3D graphics that runs in an ordinary browser. He posted the source so others can replay and modify it. Eleven Labs supplied the audio. The model placed and animated everything else on its own.

Karpathy’s stated reason for doing this is that the old informal tests are worn out. The best known is Simon Willison’s pelican test, which asks a model to draw an SVG of a pelican riding a bicycle. The subject was chosen because it is absurd enough that no finished picture of it exists in the training data, so the model has to compose it. That prompt has been more or less solved. Before it, Microsoft had GPT-4 draw a unicorn in TikZ in its 2023 Sparks of AGI paper and pointed at the result as evidence of spatial reasoning, at a time when GPT-4 had only ever seen text.

One flaw surfaced immediately. The model cannot watch its own video output. It checked its work through still screenshots, which is why parts of the animation came out wrong. Karpathy calls the whole thing rough but fun, and notes that nobody would hand build a world this way, while with a model it costs almost nothing. He sees a route to game worlds generated on demand, where you drop in as a side character.

What is behind this. There are two ways to measure a model and they answer different questions. A benchmark is a fixed set of tasks with known correct answers, so scores can be compared across models and over time. Its weakness is that once a benchmark is public, it leaks into training data and labs optimise for it, so a high score stops meaning what it used to. A vibe check is the opposite: one person gives a model something strange and looks at what comes back. It cannot be compared, it cannot be scored, and it is trivially cherry picked, since you never see the four attempts that failed. What it does catch is the thing benchmarks miss, namely whether a model can hold a long, underspecified task together without falling apart. Karpathy is explicit that this is a vibe check and not a benchmark, and that honesty is worth more than the demo.

What this means for you: treat this as a reading lesson rather than a product. When you see an impressive AI demo, the useful questions are how many attempts it took, who chose the task, and whether you could rerun it yourself. Karpathy publishing the code passes that last test, which is more than most demos manage. The practical takeaway for anyone using these tools day to day is the failure mode: the model got things wrong specifically where it could not see the result of its own work. That is the general rule. Wherever a model cannot check its output, check it yourself.

Sources

• Karpathy: the LOTR scene
• The Decoder: Unicorn, pelican, Middle-earth, Karpathy is looking for the next AI vibe test
• GitHub: Simon Willison’s pelican on a bicycle benchmark