Context compression - how long chats stay within the window

Short answer

When a chat’s history grows past ~75% of the model’s context window, Abliteration compresses the older half into a short summary and sends summary + newer half to the model instead of the full history. This happens inline, once per turn, only when needed. The summary is ephemeral - it’s used for the current turn and thrown away; the full chat history stays in the database.

Detailed explanation

Every message the model sees costs tokens in the context window. A 4096-token context fills up after roughly 15-25 message pairs of normal conversation, faster if attachments are involved. Without intervention, the model would either refuse to answer (“context exceeded”) or start dropping the earliest messages.

Abliteration handles this inline in the chat send path. Before every turn, it estimates the token count of the full history. If two conditions are met - the estimate exceeds 75% of the model’s context size, AND there are at least 4 messages - compression triggers.

How compression works:

• The chat history is split in half by message count
• The older half is sent to the model with a prompt asking for a summary of about 150 words
• The generated summary is injected as an “Earlier context summary:” system note
• The model receives [summary of older half] + [newer half verbatim] for this turn

Key properties:

• Ephemeral. The summary is not stored. Every qualifying turn recomputes fresh. The full chat history remains intact in the database.
• Once per turn. No matter how many messages, no matter how many models are involved (Council), compression runs one time per turn.
• Shared in Council. When Council mode has multiple models responding to the same message, they all receive the same compressed context - compression is not run per model.
• No component called “orchestrator.” The logic lives in the chat send path itself.

The trade-off is straightforward: compression loses fine detail (exact phrasing, small numbers in old turns) but preserves the gist (topics, decisions, entities mentioned). For most chats this is invisible. For chats where exact wording from way back matters (drafting a legal clause, working with precise numbers), the model may “forget” specific phrasing from many turns ago even though the topic is preserved.

Why it matters

Without inline compression, long chats would either error out at context exhaustion or silently degrade. With it, a small-context model (say 4K) can hold a working conversation for much longer than the raw token limit suggests - but it’s not free, and it’s not lossless.

In Abliteration specifically

Compression runs transparently - no toggle, no visible indicator that it happened this turn. It’s automatic when the thresholds hit.

Reasoning models (DeepSeek R1, Qwen QwQ) can emit <think> blocks that end up in the summarised text. The summary path does not strip these - but since the summary is ephemeral (used once, never shown, never stored), reasoning fragments don’t leak anywhere user-visible. The title generation path is separate and does strip <think>.

The full chat history - including everything that was compressed for a given turn - remains in the SQLite database and reappears in the drawer, in message list, and on chat reload. Nothing is deleted.

Related

• Context length - the ceiling compression is protecting you from
• Council mode - Council models share one compression pass
• KV cache - smaller context sent = smaller KV cache used per turn