FP8
A compact way of storing a model's numbers using eight bits each, which halves memory use compared to the older 16-bit standard.
Every weight in a model is a number, and you get to choose how many bits to spend on each one. For years the default was 16 bits. FP8 uses eight, which means the same model takes half the memory and moves through the chip roughly twice as fast, at the cost of a little precision.
Modern chips have hardware built specifically for FP8, which is why newer open models such as DeepSeek’s are often shipped in this format directly rather than being compressed afterwards. The catch is compatibility: not every card and not every serving framework handles every FP8 variant, and that mismatch is a common reason a model that runs fine on one GPU refuses to start on another.