Get the app

How Council mode manages RAM (sequential loading)

Council loads models one at a time, not all at once.

Short answer

Council loads models one at a time, not all at once. When you @-mention two models in Council mode, the first model loads, answers, unloads, then the second loads, answers, unloads. This is intentional - it keeps RAM usage low but makes responses slower than a single-model chat.

Detailed explanation

The naive way to build a “council of models” would be to load all mentioned models into RAM at the same time and have them respond in parallel. That’s fast but expensive: three 7B models loaded together eat 15+ GB of RAM, plus KV cache for each. On a typical 16 GB laptop this triggers swap and grinds everything to a halt.

Abliteration uses the opposite strategy: sequential loading. When Council mode is on and a message mentions @Llama @Qwen @DeepSeek, the app loads Llama first (2 GB RAM), streams its answer, then unloads Llama and loads Qwen (2 GB RAM), streams, unloads, loads DeepSeek, streams. Peak RAM stays at whichever single model is heaviest - never the sum.

The cost is time. Each model swap costs 2-10 seconds depending on model size and disk speed (SSD helps a lot). A three-model Council answer takes noticeably longer than a single-model chat: not 3× longer, but often 2× because of the reload overhead on top of generation.

Context is preserved across model swaps. When Qwen’s turn comes, it sees the full chat history including Llama’s earlier response. Models don’t know which model wrote earlier turns - they just see assistant: ... - but they see everything and can react to it.

Why it matters

If you have a lot of RAM (32+ GB) and want faster Council, sequential loading feels overly cautious. If you have less RAM (8-16 GB) and want stability, it’s exactly right - you can @-mention 5 different models and Abliteration won’t OOM your machine. This is a deliberate design choice, not a bug.

Concurrent loading (option to keep multiple models resident for users with lots of RAM) is on the roadmap but not implemented.

In Abliteration specifically

Model loading order in Council follows the order you typed the @-mentions. @Qwen @Llama loads Qwen first. Between each model, Abliteration shows the “Loading model…” indicator briefly, then the pulsing cat during generation.

If a Council chat gets long enough to trigger context compression, all mentioned models share the same compressed history - compression runs once per turn, shared across every model, not repeated per model.

Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Get the app Browse the catalog
Open in Abliteration