Slow generation on an older Intel Mac
Intel Macs without a fast discrete GPU run inference on the CPU, which caps out at three to ten tokens per second on a 7B model. Pick smaller models and quantize hard.
Short answer
Intel Macs run inference on the CPU. That path is honest but slow, three to ten tokens per second on a 7B model is a normal reading on a 2019 or 2020 MacBook Pro. If the wait feels unusable, drop to a model under 3B parameters and a Q4 quantization. Older Intel Macs without a modern discrete GPU are not the target rig for local models, and an Apple Silicon Mac from any generation will outrun them by a wide margin.
Detailed explanation
Why Intel Macs are slow at this. Apple Silicon chips share RAM between the CPU cores and GPU cores on the same die, so a model loaded into memory is instantly available to Metal, the graphics API the app uses for acceleration. Intel Macs do not have that architecture. The Metal backend can technically run on the AMD GPUs shipped in Intel Macs, but the memory model and driver path make the gain much smaller, so the app stays on the CPU by default. CPU inference is fine, but a Core i5 or Core i7 from 2019 cannot approach the throughput of an M-series chip on the same model.
Expected tokens per second on Intel Macs.
- Core i5 or i7 quad-core, 2018 to 2020, 16 GB RAM: 3 to 7 tokens per second on a 7B Q4 model.
- Core i7 or i9 eight-core, 2019 to 2021, 32 GB RAM: 5 to 10 tokens per second on a 7B Q4 model.
- Anything older or with an HDD: below 3 tokens per second, first load can take minutes.
For comparison, an M1 Air with 16 GB reads the same 7B Q4 file at thirty to forty tokens per second. That is the honest gap.
What actually helps on an Intel Mac.
- Pick smaller models. A 3B Q4 model runs at ten to twenty tokens per second on the same rigs that struggle to reach five on a 7B. The quality drop is real but so is the speedup.
- Quantize harder. A 7B Q3 model reads faster than the same 7B at Q5, because there is less data to move through the CPU on every token. See Quantization explained.
- Close everything else. Chrome, Slack, and the browser hosting the model chat all compete for the same RAM and cache lines. Free RAM matters. See Free vs total RAM.
- Stay on the SSD path. If the model is stored on an external HDD or a spinning-disk time machine drive, first load is very slow. Move the model file to your Mac’s built-in SSD.
What does not help on an Intel Mac. Turning Metal on (if the row is present in Settings > Hardware, which is only sometimes true on Intel) rarely changes tokens per second on the AMD GPUs shipped in these machines. Buying more RAM raises the ceiling on which models will fit, not the speed at which they run.
When to stop trying. If a 3B Q4 model on an Intel Mac still feels too slow, the machine is below the practical floor for local models. An Apple Silicon Mac from any generation, including a base-spec M1 Air, is the smallest upgrade that changes the experience. See Apple Silicon acceleration.
Why it matters
The app does not pretend an Intel Mac is fast, and the numbers on the Hardware screen match the numbers you will see during a chat. Slow generation on an older Intel Mac is not a bug and not a misconfiguration. It is the CPU path, running at the speed the CPU can run it. Knowing that changes what you download.
In Abliteration specifically
The fit-check dialog on the model catalog will still say a 7B model fits into a 16 GB Intel MacBook Pro, because it does fit. The dialog reports what will load, not what will feel fast. On an Intel Mac, treat any green light as “loads, but check tokens per second before committing to daily use”.