AMD trained a fully open model on its own chips and published everything except a commercial licence
Instella-MoE-16B-A3B has 16 billion parameters but uses only 2.8 billion per token. AMD released the weights from every training stage, the data mixtures and the code, and licensed the weights for research only.
AMD has released Instella-MoE-16B-A3B, a language model trained from scratch on its own Instinct MI300X and MI325X accelerators. The headline number is the split: 16 billion parameters in total, but only about 2.8 billion of them active for any given word the model processes.
That split is what mixture-of-experts means. Instead of running the whole network for every request, the model routes each piece of input to a small subset of specialised sub-networks, so you get the knowledge of a bigger model at roughly the running cost of a much smaller one. AMD reports a base score averaging 76.7 across its benchmark suite, ahead of other fully open models including Moonlight-16B-A3B at 76.2 and OLMo-3-7B at 70.1, with the reasoning-tuned “Think” version reaching 73.22 after post-training. Two engineering choices carry the release, a gated version of multi-head latent attention and a connectivity scheme AMD calls FarSkip-Collective, which together it credits with a 12.7 percent training speedup and 39.2 percent lower time to first token, the delay before a model starts producing output. The word “fully open” is doing real work here: AMD published the weights from every training stage, the data mixtures, the training configuration and the inference code, which is more than most open-weight releases include. The catch is the licence. The weights ship under ResearchRAIL, meaning academic and research use only, while the training codebase is MIT. This is not a model you can drop into a product.
What is behind this. Nvidia’s dominance in AI is only partly about silicon. The larger moat is CUDA, the software layer that almost every AI framework was written against, and the fact that nearly all published training recipes were developed and debugged on Nvidia hardware. AMD cannot fix that with a spec sheet. It can fix it by demonstrating, in public and in detail, that a competitive model can be trained end to end on AMD silicon with AMD’s own ROCm software stack. Publishing the intermediate checkpoints and the data mixtures is aimed squarely at researchers, because those are the people who decide which hardware the next generation of tooling gets written for. The research-only licence tells you AMD’s goal here is credibility, not market share.
What this means for you: for most readers, directly, nothing. You will not run this model and you would not want to. Indirectly it matters more than most model releases, because the cost of everything you pay for in AI is downstream of how many companies can credibly sell training hardware. A second serious supplier is the only realistic mechanism for prices to come down. For researchers and students, this is a rare thing: a competent model where you can inspect not just the finished weights but every stage of how it got there, which makes it genuinely useful for studying how training actually works.
Sources
Source: https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.html
The number of AI chips in the world is doubling roughly every nine months
Epoch AI counts about 20 million AI chips in data centres today and expects around 200 million by the end of 2028. That is the single number that explains most of what happens next.
