An 80 billion parameter model now runs on a normal Mac in 4.3 GB of memory, and a 35B one runs on an iPhone
Swiftlet is an open runtime that streams a model's expert weights from the SSD instead of loading them into memory. A Qwen model with 80 billion parameters peaks at 4.3 GB of RAM on a Mac. The trade off is speed and a large download.
A developer has published Swiftlet, an open source runtime that runs very large open models on ordinary Apple hardware by refusing to load most of the model into memory at all. The headline numbers: Qwen3-Next-80B, a model with 80 billion parameters, takes 42 GB on disk but peaks at 4.3 GB of RAM on an M5 Mac, producing about 5 words worth of text per second. A 35 billion parameter Qwen runs on an iPhone 17 in roughly 2.5 GB. The code is Apache 2.0 licensed.
The trick is the model’s architecture. These Qwen models are what is called a mixture of experts, meaning the model is split into hundreds of specialist sub networks and only a handful get used for any given word. The 80B model routes each token to 10 of its 512 experts, so only about 3 billion parameters are actually doing work at a time. Swiftlet keeps that small permanently active core in memory, roughly 2.5 GB, and fetches the individual experts from the SSD exactly when they are needed. It packs them at a fixed size so one expert equals one disk read, and caches the popular ones. Apple SSDs are fast enough that the misses barely hurt.
What is behind this. For two years the rule of thumb for running AI locally was simple: the model has to fit in your memory, so a 70B model meant an expensive machine. Expert streaming quietly breaks that rule for one specific family of models. It is not new as an idea, the author credits earlier projects TurboFieldfare and ANEMLL, but this is the first version shipped as a library, a command line tool, an OpenAI compatible local server and a finished iPhone app. The author also states the honest limitation up front, and it is the most useful sentence in the whole readme: because only about 3B parameters are active per token, these models “chat and write like large models but recall facts like small ones.” Size on disk is not knowledge in your pocket.
What this means for you: if you own an Apple Silicon Mac and have been curious about running AI on your own machine, with nothing sent to a server, this lowers the hardware bar considerably. Budget the disk space, 18 GB for the 35B and 42 GB for the 80B, and expect reading speed rather than instant answers. On iPhone the 35B currently manages about one word per second, which is a demo rather than a daily tool. If you do not tinker, the thing to take from this is directional: the gap between what a data centre runs and what your laptop runs keeps narrowing, and privacy friendly local AI is getting harder to dismiss as a hobby.
Sources
A record eight Pulitzer winners and finalists disclosed using AI, and what they used it for is the interesting part
Eight Pulitzer awardees declared AI use this year, the most since disclosure became mandatory in 2024. In almost every case the tool was used to search or translate large document piles, not to write.
