Swiftlet
An open source runtime that runs very large mixture-of-experts models on Macs and iPhones by streaming most of the model from the SSD.
Swiftlet is a free, Apache 2.0 licensed piece of software for Apple hardware, published in August 2026. Normally a model has to fit into your memory to run at all. Swiftlet keeps only the permanently active core of a mixture-of-experts model in memory and fetches the individual expert sub-networks from the SSD exactly when a word needs them.
The result is that an 80 billion parameter Qwen model takes 42 GB of disk space but peaks at about 4.3 GB of RAM on a Mac, and a 35 billion parameter one runs on an iPhone. The trade-off is speed, and the honest caveat from its author is that models used this way write like large models but remember facts like small ones.