Context length - what the number means

Short answer

Context length is the model’s working memory in tokens - how much text it can see at once. 4096 tokens ≈ 3000 English words ≈ 6 pages. When you exceed it, older messages start being forgotten (or summarized).

Detailed explanation

Every message in your chat costs tokens. When the total exceeds the context length, something has to give. Older content either drops off, gets summarized, or the model refuses to answer.

Token conversions (rough):

• 1 token ≈ 0.75 English words
• 1 token ≈ 0.5 Russian words (Cyrillic is heavier in tokens)
• 1 token ≈ 0.5 characters in Chinese or Japanese
• 4096 tokens ≈ 3000 English words ≈ 6 pages of text
• 32,768 tokens (32K) ≈ 24,000 words ≈ ~50 pages ≈ a short novella
• 131,072 tokens (128K) ≈ ~200 pages ≈ a full book

What eats context:

• Every user message + every assistant reply
• System prompts
• Attachments (a 100 KB PDF can be 15,000+ tokens)
• Reasoning models’ <think> blocks (large - sometimes bigger than the answer itself)

Two context numbers matter:

1. n_ctx_train - what the model was trained on. This is the ceiling. Qwen 2.5 3B: 32K. Llama 3.1 8B: 128K. DeepSeek R1: 164K.
2. Current context - what you’ve allocated for this chat. Can be lower than n_ctx_train (to save RAM) but not higher (the model will produce nonsense above what it was trained on).

Higher context = more KV cache = more RAM. See the KV cache article for the math. On tight RAM, dropping context from 32K to 8K can free 3-4 GB.

Why it matters

Too low context = model forgets quickly, chat feels like Groundhog Day. Too high context = eats RAM that could have gone to a bigger/better model. The sweet spot depends on your use case: casual chat is fine at 4K-8K, long documents or Council mode benefit from 16K-32K.

In Abliteration specifically

Every model shows two context numbers:

• Its trained ceiling (from the GGUF header, n_ctx_train)
• The currently allocated context (Auto by default, or Manual via the slider)

The per-model context slider (⋯ → Edit → context) lets you override the default. The slider maxes out at the model’s trained ceiling - you can’t go higher.

The Model info tooltip shows the current context in tokens plus rough conversions to pages/words in English, so you have intuition for what “4096 tokens” actually holds.

When context compression kicks in (once the chat’s estimated token count exceeds ~75% of the model’s context window), the older half of the chat gets summarized on the fly. This means practical usable chat length is longer than the raw token limit suggests.

Attachment token counts appear under each file in the input strip (color-coded: green under 50% of context, amber 50-90%, red above 90%) so you can spot before sending that a big PDF is about to blow past the limit.

Related

• n_ctx_train - the trained ceiling
• Context slider - the manual override
• KV cache - the RAM cost of context
• Orchestrator summarization - how long chats stretch further