← Glossary Term
Megakernel
One large GPU program that does the work of many small ones, so the chip spends less time waiting between steps.
A kernel is a small program that runs on a graphics chip. Running a language model normally means launching lots of these small programs one after another, and each hand-off wastes a little time. A megakernel fuses many of those steps into one big program, so the chip can keep working without stopping to be told what to do next.
The payoff is speed, especially when you run a model on your own machine and every millisecond counts. The catch is that megakernels are hard to build and are usually tuned for specific chips, so a trick that works on one computer may not help on another.
Mentioned in