Someone got DeepSeek V4 Flash running in production on a single AMD card and published every patch it took
A public repository documents the configuration, patches and tuning needed to serve DeepSeek V4 Flash on one AMD MI300X, unquantised. It is a useful record of how much work running an open model on non Nvidia hardware still is.
DeepSeek released the weights for V4 Flash on 31 July under an MIT licence, which means anyone can download and run it. A repository published this week shows what “anyone can run it” costs in practice: the full set of configuration files, patches and tuning tables one engineer needed to serve the model reliably on a single AMD MI300X card, in production, at the checkpoint’s original precision with no extra compression or offloading to system memory.
The repository is unusually honest about the work involved. It lists the Docker Compose stack, file overlays pinned by their SHA-256 checksums so nothing drifts, reference diffs against the upstream projects, and the tuning tables. The fixes cover the model’s FP8 number format, expert routing under high concurrency, causal speculative verification, synchronisation between CPU and GPU memory, and several kernel shapes that nobody had tuned for this card. Related bug reports in the vLLM and SGLang projects show the same rough edges from the other direction, including a vLLM release whose notes claimed AMD support that did not hold up on the newer MI350X.
What is behind this. Nvidia’s real advantage has never only been the silicon, it is CUDA, the software layer that everything in AI is written against. AMD’s equivalent, ROCm, has improved a great deal, but “the weights are open” and “the weights run on your hardware” remain different statements, and the gap is filled by exactly this kind of unglamorous engineering. Publishing it matters more than it sounds: pinned patches and reproducible configs are how a one off heroic bring up turns into something the next person can copy in an afternoon. Every one of these repositories makes the second GPU vendor slightly more viable, which is the only thing that puts downward pressure on compute prices.
What this means for you: if you rent GPUs or run inference for a product, AMD cards are usually cheaper per unit of memory and this is a concrete data point on what it takes to use them, including the failure modes to expect. If you do not, the story is still worth knowing as background. Nearly all of the cost of AI today traces back to one company’s chips and one company’s software stack, and everything that erodes that dependency eventually shows up in what you pay for a subscription. Worth keeping expectations grounded though: this is one engineer’s setup for one card and one model, not a general promise that open weights run anywhere.
Sources
Source: https://github.com/ryanzhou/deepseek-v4-flash-mi300x
Researchers asked why chatbots are bad at spreadsheets, and the answer is stranger than expected
A new paper tests five explanations for why large language models lose to decades old methods on tabular prediction. In low dimensions the model behaves like a simple distance based classifier. In higher dimensions nothing classical explains it.
