Lithos-Metal Is a New Open Engine for Running AI Fast on Apple Silicon Macs
The open-source lithos-metal engine targets Qwen models on Apple's M5 chips and cites 168 tokens per second. That figure is a projection under stated assumptions, not a measured end-to-end speed.
A new open-source project called lithos-metal aims to make large language models run faster on Macs. It is an inference engine, meaning the program that actually runs a trained model and produces its answers, built specifically for Apple silicon and licensed under Apache 2.0. A Hacker News post highlighted a headline number: the Qwen3.8-27B model at 168 tokens per second on a 40-core M5 Max. A token is a small chunk of text, roughly three quarters of a word, so that would be very fast reading speed.
Read the fine print before getting excited. The project’s own page calls 168 a “target-compute projection” for a short 128-token context, on a 48 GB machine, assuming six accepted tokens per eight-row verification pass and leaving out some overhead such as drafting and rewinding. It is not a measured end-to-end speed, and the page warns that timing scopes differ between backends, so it is not a like-for-like comparison with other tools. At a 32,000-token context the projected figure drops to 127 tokens per second.
The speed ideas are interesting. “Megakernels” fuse many small GPU steps into one larger program per layer, cutting waiting time. Generation control stays on the GPU instead of bouncing back to the CPU. And a method called DSpark uses speculative decoding: a small, quick draft model guesses several tokens ahead, and the big model checks all of them in one pass, keeping the correct ones. When the guesses are good, you get several words for the price of one.
Where it stands: the project is young. It needs an Apple silicon Mac on macOS 26 or later and enough unified memory for the model, the draft model and the context. Measured recipes exist mainly for the M5 Max with 40 GPU cores and the M5 Pro for smaller models, while other chips are unmeasured and get no automatic fusion. Only one generation runs at a time, and some model integrations are labelled experimental. It also offers a local server compatible with common chat interfaces and can launch coding agents.
What this means for you: if you are just starting with local AI, you do not need this; a friendly app like Ollama or LM Studio is the easier road. If you own a recent high-end Mac and enjoy tinkering, it is worth bookmarking, but wait for independent speed tests. Today’s lesson is a good habit: when a speed number arrives without “measured” next to it, ask what it is projecting.
Sources
Critical Flaw in Archon Coding Agent Lets Strangers Take Over Your Machine
A maximum-severity-class bug (CVSS 9.8) in the Archon AI coding agent exposes an unauthenticated control interface. No patch exists yet; the interim fix is to lock the interface to your own computer.
