← Glossary Term
N-gram
A run of N consecutive words or tokens, the simplest way of capturing which pieces of text tend to follow each other.
An n-gram is just a short sequence: “New York” is a 2-gram, “New York City” a 3-gram. Before neural networks, language models were built almost entirely by counting these, asking which word most often follows a given pair. It is crude, but it captures a surprising amount of how language actually behaves.
The idea never fully went away. Modern models sometimes bolt an n-gram component onto the neural part as a cheap memory for frequent phrases, which is why a 2026 model card can quote both a parameter count and a separate block of n-gram embeddings.
Mentioned in