DeepSeek Releases DeepInfer, the Engine It Uses to Serve Its Cheapest Model
DeepSeek open-sourced the serving software behind V4.1-Flash under Apache 2.0. No speed numbers yet, but it could lower the cost of running the model yourself.
DeepSeek has published DeepInfer, the software it says it uses to serve its V4.1-Flash model to customers. According to The Decoder on October 20, the code is released under the Apache 2.0 licence, which allows free use, including in commercial products. It is built on vLLM, a popular open-source serving tool, and adds faster compute routines, ways of spreading the work over many chips, and quicker text-generation techniques.
DeepSeek also released a companion called deepseek-recipe. Its job is to reproduce the way DeepSeek’s official API processes text, so that a model you host yourself behaves the same as the one you would rent. The company says code for its future models will be open as well. Its stated goal, posted on X, is “to help the community lower compute costs, reduce energy use, and make AI more affordable.”
Here is the honest part: the announcement comes without benchmark numbers, speedups or cost comparisons. The idea that DeepInfer makes V4.1-Flash cheap to run is DeepSeek’s framing, not something measured in the coverage we found. The Decoder’s view that this adds price pressure on Western providers, and may be politically motivated, is the author’s analysis and not something DeepSeek has said.
What’s behind it: “inference” is simply the act of using a trained model, as opposed to training it. Every time you send a message, a server does inference. Training happens once and costs a fortune; inference happens billions of times, so shaving a little cost from each request adds up. A serving engine is the traffic controller that decides how requests are batched, which chip handles which piece, and how fast words come back. Two companies can run the same model and pay very different bills depending on this software. By publishing its own, DeepSeek hands everyone else the tuned version it keeps for itself.
That also explains why a company known for cheap models cares. The model weights were already open; the missing ingredient for outsiders was the efficient way to run them. DeepInfer closes part of that gap.
What this means for you: if you only chat with AI in an app, nothing changes. If you or your company host open models, DeepInfer is worth bookmarking, but wait for independent tests before moving anything important. One practical caution: check your organisation’s rules on using DeepSeek products, since some governments and employers restrict them, and running software from any source deserves a security review first.
Sources
Mercury 3 Reviewed: A Chatbot That Writes Its Answer in Chunks, Not Word by Word
Inception's Mercury 3 is a diffusion language model claiming over 1,000 tokens per second at low prices. A hands-on review finds it fast and cheap, with rough edges.
