First load of a model takes forever

Short answer

Long first load usually means Windows is swapping other apps to disk to make room for the model. Once done, generation runs at normal speed. If loading is always this slow - even after a fresh restart - the model may be too big for your total RAM.

Detailed explanation

Normal first-load times (fresh boot, plenty of free RAM):

• 3B Q4 (~2 GB file): 1-3 seconds
• 7B Q4 (~5 GB file): 3-8 seconds
• 13B Q4 (~9 GB file): 6-15 seconds
• 30B MoE (~20 GB file): 15-40 seconds

These assume SSD storage. HDD roughly doubles them.

Long first-load times (many apps open, tight free RAM):

If free RAM is less than the model’s size but total RAM is enough, Windows swaps other apps’ memory pages to disk (pagefile.sys) to make room. First load can then take 30-120 seconds instead of 3-15. Once done, the model runs at normal speed - but Chrome, Slack, etc. will feel sluggish for a bit as their pages get pulled back from disk.

Fix: close some apps before first load. Second and subsequent loads of the same model in the same session are much faster (usually cached).

Truly stuck (5+ minutes with no progress):

If load never completes:

• Model file may be corrupted (partial download)
• Model file may be larger than total RAM (system can’t hold it even with full swap)
• Rare: Abliteration crash mid-load (restart the app and try a different model)

Fix: cancel, verify file size matches Hugging Face, re-download if suspicious, or pick a smaller model / lighter quant.

Cold vs warm loads:

• First load after fresh restart = cold. Slowest.
• Second load of same model after unload = warm. Faster (Windows file cache still has parts of it).
• Reload after 30 min of other work = cold again. Windows may have evicted the cache.

This is normal OS behavior, not an Abliteration issue.

Why it matters

“Slow load” and “slow generation” are different problems. Slow load = one-time cost, generation still fast. Slow generation = every token painful. Distinguishing them changes the fix (close apps once vs pick different model).

In Abliteration specifically

Model loads happen in the background - the app doesn’t freeze during load. You’ll see a “Loading {Model Name}…” indicator (naming the specific model, e.g. “Loading Qwen 2.5 7B…”) until it’s ready to generate. A pulsing cat animation appears alongside the label.

Council mode reloads models between turns (sequential loading), so each model swap in Council pays the load cost again. That’s intentional to keep RAM usage low. If Council feels slow because of this, either use fewer models per Council chat or wait for concurrent loading (roadmap).

Related

• RAM requirements - the underlying math
• Free vs total RAM - why swap-to-disk happens
• Slow generation - the different, ongoing problem