Which model size fits which rig
Rule of thumb, Q4 quantization, 8K context, comfortable fit:
Short answer
Rule of thumb, Q4 quantization, 8K context, comfortable fit:
- 8 GB total RAM → 3B models
- 16 GB → 7B models
- 32 GB → 13B or 30B (MoE)
- 64 GB → up to 70B
Free RAM matters more than total. If lots of apps are open, drop one tier.
Detailed explanation
8 GB total RAM (typical budget laptop):
- Recommended: 3B Q4 models (Llama 3.2 3B, Qwen 2.5 3B, Gemma 3 4B)
- Close everything else while running - you’ll have ~5 GB free after Windows, just enough
- 7B will swap heavily. Only try if you can close browser, IDE, everything
16 GB total RAM (mainstream laptop):
- Sweet spot for 7B Q4 models (Llama 3 8B, Qwen 2.5 7B, Mistral 7B, DeepSeek 7B)
- 13B possible if you close other apps (Chrome+Slack together eat ~5 GB)
- 3B if you want max speed with everything else running
32 GB total RAM (developer laptop):
- 13B Q4 comfortable (Qwen 2.5 14B, DeepSeek 14B)
- 30B MoE excellent (Qwen3-30B-A3B - 20 GB file, ~2B active, ~60 tok/s)
- 30B dense possible but painful (~2 tok/s on CPU)
- 70B won’t fit
64 GB total RAM (workstation):
- 30B dense comfortable
- 70B Q4 needs ~45 GB, works but leaves little room
- Multiple 7B models in Council mode possible
MoE exception: MoE models take RAM by their file size (all experts loaded) but run at active-parameter speed. A 30B MoE on a 32 GB laptop feels like a fast 3B, not a slow 30B. Practical value/RAM ratio is much better than dense at the same size.
Reasoning model note: DeepSeek R1, Qwen QwQ generate large <think> blocks (often 500-2000 tokens) before answering. This means a 4K context fills up faster than with regular chat. Give reasoning models 8K+ context if you have the RAM.
Why it matters
Picking the right size = fast, comfortable use. Picking a size too big = swap to disk, 0.5 tok/s, frustration. Picking too small = model dumb enough that you gain nothing over online alternatives.
In Abliteration specifically
The Hardware card’s Line 3 gives your personalized recommendation based on your actual free/total RAM. Green tier fits now, amber fits after closing apps, red doesn’t fit.
The fit-check dialog on model add uses the exact file size (not just tier) so a lightly-quantized 3B and a heavily-quantized 7B can both fit into the same “3B tier” of your machine.
Related
- How to evaluate for your rig - deciding process per model
- Quantization - how to squeeze a bigger model into less RAM
- MoE vs dense - why 30B MoE on 32 GB laptop works
- RAM requirements - the underlying formula