Free vs total RAM - what's the difference?

Short answer

Total = physical RAM installed in your PC. Free = what’s not currently used by anything (Windows, browser, apps). Free is what actually determines whether a model loads smoothly right now; total determines whether the model can theoretically run at all.

Detailed explanation

Take a 32 GB laptop. Windows itself uses ~2 GB. If Chrome, Slack, VS Code and Spotify are open, that’s easily another 8 GB. So free = 32 - 2 - 8 = 22 GB. If you open a 30 GB Abliteration model in this state, Windows has three options:

Option 1 - Fits in free (best case). Model loads directly into RAM. Everything runs at normal speed. This is what you want.

Option 2 - Fits in total but not in free. Windows quietly swaps parts of Chrome, Slack, etc. to disk (pagefile.sys) to make room. Model loads, runs at normal speed once loaded. But the first load takes longer (10-60s instead of 2-3s), and the swapped-out apps feel sluggish when you switch back to them.

Option 3 - Doesn’t fit in total. Windows has no choice but to run the model itself half-in-RAM half-on-disk. Every token generated requires reading gigabytes from SSD. Speed drops to 0.1-0.5 tok/s. Practically unusable.

Abliteration’s fit-check dialog distinguishes these cases. If needed RAM is more than free but less than total, you get a message about closing other apps for best performance. If needed is more than total, you get a stronger warning that it will be very slow.

Why it matters

“32 GB total” sounds like a lot but if 27 GB is already in use, you effectively have 5 GB for the model. The number that matters right now is free, not total. This is why the same PC can happily run a 13B model in the morning and fail at loading it in the afternoon after you’ve opened a bunch of tabs.

In Abliteration specifically

The Hardware card shows the two numbers on separate lines: Line 1 - Your PC: ..., 32 GB RAM, ... (total installed) - and Line 2 - Free right now: 5.3 GB (available now). Line 3’s “You can run” recommendations use both:

• Green tier - fits in free (runs now)
• Amber tier - fits in total with ~6 GB headroom (close apps first)
• Red tier - doesn’t fit total (won’t work)

The fit-check dialog on model add shows both numbers when a model is too big for free RAM, so you can decide whether closing Chrome is enough.

Related

• RAM requirements - how much a model actually needs
• Fit-check dialog - the specific warnings tied to these thresholds
• First load is slow - usually option 2 above