license: apache-2.0
tags:
- benchmark
- apple-silicon
- mlx
- gguf
- llm-inference
- tokens-per-second
- local-llm
pretty_name: Apple M5 Pro 24GB — Local LLM Tokens/Sec Benchmarks (12 models)
Apple M5 Pro (24GB) — Local LLM tokens/sec benchmarks
Real-world tokens/sec numbers for 12 local coding/agentic LLMs, measured on a specific,
commonly-owned but previously unbenchmarked configuration: Apple M5 Pro, 24GB unified
memory. At the time of writing, no public benchmark existed for this exact chip + RAM
combination for these models. Includes both the official newly-open-sourced Qwen3.8-27B and
a popular uncensored/abliterated finetune of it, for comparison.
Hardware
- Chip: Apple M5 Pro (15-core: 5 Efficiency + 10 Performance)
- Unified memory: 24 GB
- OS: macOS 26.6.2 (build 25G83)
- GPU memory ceiling: default macOS
iogpu.wired_limit_mbis0(auto, ~18GB on this
machine); some models below required raising it to22000(22GB) to load at all — noted
per-row.
Methodology
- Backend: MLX (
mlx_lm.generate) for MLX-format models,llama.cpp(llama-bench+llama-cli) for the one GGUF model. - Standard metric (
std_prompt_tps/std_gen_tps): a short synthetic prompt ("Write a
one-paragraph summary of how binary search works."), capped at 128 output tokens — directly
comparable to thepp/tgconvention already used across HF model cards and llama.cpp
benchmark threads. - Real-task metric (
codingtask_*): a fixed, non-trivial coding prompt — implement a
thread-safe concurrent restaurant waitlist system as a FastAPI REST API — capped at 1500
output tokens. This measures throughput under a realistic agentic-coding workload, not just
a synthetic string. - Single run per model, temperature 0, no warm-up discard. Every model was benchmarked
with all background downloads finished and Chrome/Safari closed, to avoid contention
skewing the numbers (an earlier download-contaminated pass is kept separately inresults_with_downloads_active.csvin this repo for reference — it shows a real, measurable
effect from background downloads on tok/s). - Correctness was checked, not just speed — the notes column flags any model whose
coding-task output was garbage (repetition loops, unfinished reasoning) despite a valid
tok/s number. A high tok/s on broken output is not a useful result.
Results
| Model | Quant | Size (GB) | Backend | Coding-task gen tok/s | Coding-task prompt tok/s | Result |
|---|---|---|---|---|---|---|
| DeepSeek-Coder-V2-Lite-Instruct | 4bit | 8.2 | mlx | 125.8 | 363.9 | ✅ correct, working code |
| Qwen3-Coder-30B-A3B-Instruct | 3bit | 12.4 | mlx | 105.2 | 187.6 | ✅ correct, working code |
| Qwen3-Coder-30B-A3B-Instruct | 4bit | 16.0 | mlx | 94.1 | 97.2 | ❌ degenerated into repeated imports |
| GLM-4.7-Flash | 4bit | 15.7 | mlx | 73.1 | 106.3 | ⚠️ reasoning model, never left chain-of-thought |
| Qwen2.5-Coder-7B-Instruct | 4bit | 4.28 | mlx | 64.8 | 1037.3 | ✅ correct, working code |
| Seed-Coder-8B-Instruct | Q4_K_M | 4.7 | llama.cpp | 53.7 | 852.0 | ✅ correct, working code |
| Qwen3.5-9B | 4bit | 7.6 | mlx | 44.2 | 658.2 | ❌ repetition-loop failure |
| Qwen2.5-Coder-14B-Instruct | 4bit | 7.7 | mlx | 32.8 | 249.0 | ✅ correct, working code |
| Qwen3.6-27B | 4bit | 18.6 | mlx | 15.0 | 14.5 | ⚠️ needed raised GPU memory ceiling; cut off mid-function |
| Qwen3.6-35B-A3B | 4bit | 23.0 | mlx | — | — | ❌ does not run — thrashes the whole machine even with raised GPU ceiling |
| Qwen3.8-27B (official) | 4bit | 15.0 | mlx | 18.1 | 126.0 | ⚠️ reasoning model, hit cap entirely in planning, never wrote code |
| Huihui-Qwen3.8-27B-abliterated (uncensored) | Q4_K | 15.65 | llama.cpp | 15.0 | 187.4 | ⚠️ same behavior as official — reasoning-only, no code output |
Full data with all columns (peak memory, standard-prompt numbers, exact notes) is inresults.csv.
Key findings
- Quantization level affects correctness, not just speed. Qwen3-Coder-30B-A3B's 3-bit
quant produced correct code while the 4-bit quant of the same model degenerated into a
repetition loop on the identical prompt — a smaller/lower-precision file was the more
usable one here. - 24GB has a hard ceiling around ~20GB models, independent of macOS GPU memory tuning.
Qwen3.6-35B-A3B (23GB on disk) could not be coaxed into running even after raisingiogpu.wired_limit_mbto 22000 — it caused severe system-wide memory thrashing instead of
a clean out-of-memory error. - Background activity (downloads) measurably affects tok/s — see
results_with_downloads_active.csvfor the same models benchmarked while another model was
still downloading in the background. - MoE models are not automatically fast on this hardware. Despite having only ~3B active
parameters, Qwen3-Coder-30B-A3B's total 30B footprint still puts real memory pressure on a
24GB machine, and Qwen3.6-35B-A3B (35B total) doesn't run at all. - Newer ≠ better for this task. Qwen3.8-27B (both official and uncensored) is a
reasoning-first model that spent its entire 1500-token budget on chain-of-thought planning
and never emitted actual code — despite being the newest model in this set, older
non-reasoning models like DeepSeek-Coder-V2-Lite completed the task correctly and faster. llama.cpp's default--ctx-size 0(load full model context) can silently cause a Metal
out-of-memory error on large models even when the model weights themselves fit comfortably
— it allocates KV cache for the model's full trained context length regardless of actual
prompt length. Explicitly bounding context (-c 4096here) fixed it.
Files
results.csv— the clean, final benchmark pass (all downloads finished, no background
contention). This is the authoritative data.results_with_downloads_active.csv— an earlier pass on the same models, run while other
downloads were active in the background. Kept for comparison, not the primary result.