Apple Silicon acceleration
On Apple Silicon Macs the app runs inference through Metal on the GPU cores, which is two to five times faster than the CPU path used on Intel Macs and on Windows.
Short answer
On an Apple Silicon Mac (M1, M2, M3, M4) the app runs inference through Metal, which uses the GPU cores of the chip. On the same 7B model that a same-year Intel Mac reads at five to ten tokens per second on CPU, an M-series Mac reads at fifteen to forty. The switch is automatic. A toggle in Settings under Hardware lets you turn it off, though there is almost no reason to.
Detailed explanation
What Metal is doing. Metal is Apple’s low-level graphics and compute API. The llama.cpp runtime the app is built on has a Metal backend that offloads matrix multiplication, the core work of running a language model, from the CPU cores to the GPU cores. On an Apple Silicon chip, CPU and GPU sit on the same die and share the same pool of RAM, so there is no data-copy penalty for moving a model onto the GPU. The chip effectively becomes a very large, very fast neural coprocessor with the whole system RAM available to it.
Why it is fast on Apple Silicon and not on Intel Macs. The Metal backend can run on the discrete AMD GPUs shipped in Intel Macs, but the performance gain is smaller, and the memory model is different, so Intel Macs stay on the CPU path by default. Apple Silicon’s unified memory is what turns “a bit faster” into “several times faster”, because there is no separate GPU memory that a model has to be shuttled into.
When the toggle exists and what it does. Settings > Hardware shows a Metal acceleration row on Apple Silicon Macs. It is on by default. Turning it off puts the app on the CPU path for everything, which is useful only for a narrow debugging case, for example when a specific model behaves oddly under Metal and you want to check whether the CPU path shows the same behaviour. On Intel Macs and on Windows the row is absent.
The numbers you should see. A 7B model at Q4 quantization on the CPU cores of an M1 Air produces around eight to twelve tokens per second. The same file with Metal on turns into thirty to forty tokens per second. A 13B Q4 model moves from four or five tokens per second on CPU to fifteen to twenty on Metal. The exact multiplier depends on the model and the quant, but two to five times is the usual range. See Expected tokens per second for the wider table.
Why it matters
Local models on a laptop are usually judged in seconds waited per answer. Two seconds to first token and thirty tokens per second is a chat experience you would actually use. Fifteen seconds to first token and six tokens per second is a demo. On Apple Silicon, Metal is what separates the two.
In Abliteration specifically
The app decides once, at launch, whether the machine has an Apple Silicon chip and enables Metal accordingly. There is no per-model setting, the choice applies to every model you load. If you flip the toggle off and want to test the difference, load the same model twice, once with the toggle in each position, and compare the tokens-per-second reading during a response.
The Hardware screen names the specific chip, for example “Apple M2 Pro”, underneath the RAM row. That name comes from the same detection pass that decides whether Metal is available.