My attachment eats all the context

Short answer

Big documents burn context fast. Options: (1) increase the model’s context via the slider, (2) trim the document before attaching, (3) split into pieces and attach one at a time, (4) switch to a model with a bigger trained context ceiling.

Detailed explanation

How to tell it’s a problem:

When you attach a file, a chip appears with a border color:

• Default (neutral) border - under 50% of context. Fine.
• Amber border, ⚠ icon top-right - 50-90% of context. Tight but workable.
• Red border, ⚠ icon top-right - over 90%. Some content will be truncated.

The per-chip percent + warning text appear on hover (tooltip). Under the whole attachment strip, an aggregate badge shows the total across all attachments: Attachments: ≈N tokens (X% of context).

Options to fix, in order of least effort:

Option 1 - Bump the context slider.

If your model was trained for a bigger context than what’s currently allocated, use the slider. Models tab → ⋯ Edit → context → Manual → drag up. Watch the live RAM cost - you need to have that RAM free.

Example: Qwen 2.5 7B is trained for 128K but might be set to 4K by default. Bumping to 32K solves most document-attachment problems, at a modest RAM cost.

Option 2 - Trim the document.

Open the file, delete irrelevant sections, save as a smaller file, re-attach. Especially effective for PDFs where large portions are boilerplate, appendices, references - usually the interesting content is a few pages.

Option 3 - Split and attach in turns.

Break a 100-page PDF into 3-4 files, attach one per message, ask about that section only, then move on. Uses context sequentially, and once older messages push the chat past ~75% of context, they get compressed automatically.

Option 4 - Switch to a bigger-context model.

If a specific document workflow always blows past context, use a model trained for big context:

• Qwen 2.5 (any size) - 128K trained context
• Llama 3.1 (any size) - 128K
• DeepSeek R1 - 164K
• Some Gemma 3 variants - 128K

Older models (Llama 2, Mistral 7B v0.1) top out at 4-8K - bad choice for document work.

Option 5 - Accept truncation.

If the document is huge and the specific part you care about is at the beginning, sometimes just letting the tail get truncated is fine. Watch the traffic light - if the first half fits comfortably, that first half is what the model reads.

Why it matters

Attachments are the fastest way to blow through context. Understanding the traffic light and having a few strategies to shrink usage keeps you productive with big documents instead of stuck at “context exceeded” errors.

In Abliteration specifically

Token counts on chips are estimates, not exact. They’re within ~20% of reality. If a chip says 40% context, don’t worry - you have headroom. If it says 85%, be cautious about long replies (the model needs some of that context for its own response).

Once the chat’s token estimate crosses ~75% of the context window, Abliteration compresses the older half of history into a short summary - this reclaims context space. In a long chat where the first attachment set the context high, later turns get a bit more room as older messages are compressed.

Related

• Attachment traffic light - reading the color-coded chip
• Context length - the underlying concept
• Context slider - how to bump context per model