inference

Inference is the act of using a trained AI model, feeding it a prompt and getting a response. It is distinct from training, which is the far more expensive, one-time process of building the model’s weights in the first place.

The distinction is useful for understanding AI costs. Training a frontier model can require months of work and enormous compute, but that cost is paid once. Inference is paid every time someone uses the model, which is why providers meter it, usually by the token, and why efficiency improvements that make inference faster or cheaper matter so much.

It is also why running smaller models locally is practical: even a capable laptop can handle inference for many tasks, even though it could never have trained the model itself.