Foundation model
A large, general-purpose model trained once on broad data and then adapted to many different tasks.
A foundation model is the big, expensive, general model that everything else gets built on top of. Instead of training a separate system for translation, another for summarising and another for code, you train one very large model on an enormous amount of text or images, and then adapt it to specific jobs with a little extra training or just clever prompting.
The term became popular because it captures the economics. Building one costs enormous amounts of compute and data, which is why only a handful of organisations do it, while thousands of companies build products on top. When you read that a lab is “training a new foundation model”, that is the expensive part of the industry being described.
-
Researchers asked why chatbots are bad at spreadsheets, and the answer is stranger than expected
-
Amazon quietly stops developing most of its own AI models
-
ChatGPT can now read your health records, which is handy and worth thinking about
-
Spending on AI models is set to hit $64 billion this year, up 63%
-
SAP Just Spent Over a Billion Euros on AI That Reads Spreadsheets, Not Chats
-
Google's SensorFM Wants to Be One AI Brain for All Your Wearable's Health Data
-
Stanford's Big Yearly AI Report Card Is Out, and the Trust Gap Is Widening