license: apache-2.0
tags:

• abliteration
• heretic
• benchmark
• gguf
• quantization
  language:
• en

Does abliteration make models cheaper to run? A measured answer.

A controlled study of what abliteration (directional refusal removal) actually costs or
saves on real agentic coding work, measured on two ~30B open-weight models at matched
quantization, with a measured noise floor and every arm repeated.

Headline result: it depends entirely on the model, and the sign flips.

Same procedure, same suite, same quantizer, opposite outcomes. Every number is the mean of
two independent runs.

Why this study exists

Abliterated models are usually promoted as "uncensored." That framing is not measurable and
not why most people run them. The testable claim is different and more interesting:

Refusal training installs an alignment tax — hedging, preamble, disclaimers,
restating the question — that applies to all output, not just to refusal-adjacent
requests. Removing the refusal direction removes the tax, so the model answers the same
question correctly in fewer tokens.

If true, abliteration is a performance optimization, and its benefit should be visible
on coding tasks that never trigger a refusal at all. That is exactly what we tested.

Result 1 — Glimmer: a third fewer tokens, identical correctness

−35.8% output tokens. Scores unchanged: 9/9 and 142/142 test cases on all four runs.

The effect is not carried by one task. The abliterated arm used fewer tokens on 8 of 9
tasks (sign test p ≈ 0.02), with the largest drops on btree_insert_delete (13,286 → 3,458,
−74%) and mini_sql_executor (5,623 → 1,726, −69%).

Result 2 — Qwen: more tokens and a reproducible capability loss

+18.6% output tokens, fewer tokens on only 2 of 9 tasks, and a failure that repeats:

btree_insert_delete    qwen-stock   PASS  PASS
                       qwen-ara     FAIL  FAIL

Stock clears that task twice; the abliterated build fails it twice, by two different routes
(a timeout, then a run that finished but produced no artifact). Glimmer clears it in all
four of its runs, so the task itself is not the problem.

Result 2b — method determines harm, model determines benefit

A third Qwen arm settles the method-vs-model question. darkc0de/Qwen3.8-27B-heretic uses the
same blunt rank-1 heretic v1.4.0 that worked on Glimmer, at a much lower dose (KL 0.0095,
refusals 89 → 64).

Compared over the 7 tasks every Qwen run passed — the only fair basis, since summing
across arms that completed different task counts flatters whichever arm failed more:

And on capability, across every run at three timeout settings:

* the 8/9 was a 2,400 s timeout on backtracking_regex; at a 5,400 s budget the same build
scores 9/9. Opening the wall also proved ARA's failure is not a timeout artifact — with
90 minutes available it fails btree in 1,512 s, producing 16,584 tokens and no artifact.

This splits the two hypotheses cleanly:

• Method determines harm. Surgical arbitrary-rank ablation costs tokens and reproducibly
  destroys one task. Blunt rank-1 ablation is neutral on tokens and preserves 9/9.
• Model determines benefit. Blunt ablation buys Glimmer −35.8% but Qwen only −2.6%. The
  removable alignment tax exists in one model and not the other.

Caveat: dropping the two hardest tasks removes the stable bulk, so this subset is noisier
than the 9-task totals — stock spreads 16% here versus 0.8% over all nine, and ARA's +19.2%
leans on one high run. The darkc0de figure (two runs within 1.8%) is the solid one; ARA is
better stated as "higher, with wide spread" than as a precise percentage.

Why the sign flips — the leading hypothesis

The two models were abliterated by different methods, and the difference is instructive.

The more sophisticated method, achieving complete refusal removal at lower KL,
produced the worse model. That is backwards under any "better ablation is better" story.

The hypothesis that fits: the alignment tax and the refusal behaviour share the dominant
direction. A blunt rank-1 ablation deletes that whole direction and takes the hedging with
it as collateral. Arbitrary-rank ablation can resolve finer structure, so it finds the
minimal precise intervention that zeroes refusals — and leaves the tax untouched, because
nothing in the objective asks for it.

Note what heretic optimizes: it co-minimizes refusals and KL divergence. If the tax
removal is the benefit, then minimizing KL is minimizing the repair. The better a method
is at its stated objective, the less of the thing we actually want it delivers.

A further wrinkle: heretic's refusal detector is keyword-based on response text. A model
that emits "I can't help with that" and then helps is scored as a refusal despite
complying. Part of what was driven to 0/100 may have been refusal text in front of
compliant answers — a surface artifact, optimized at real KL cost.

This remains a hypothesis. Method and model are confounded: each model got one method.
An arm testing stock heretic on Qwen is in progress.

Result 2c — the benefit attenuates at lower bit depth

Run on a second machine (RTX 4060 Ti, llama.cpp 84e908c62, spec-protected harness), n=2 per arm:

Direction preserved, magnitude cut by roughly two thirds. Arm spreads are 7.3% and 9.2% at
n=2, so the standard error on the delta is ~6% — this is a ~2σ result. State it as
"attenuated, direction preserved, magnitude not well determined", not as −13.2%.

A trap this exposes, which applies to nearly every abliteration comparison published:
stock is not fixed across quants. It went 54,044 → 59,768 (+10.6%) from Q4_K_M to IQ3_M.
Anyone comparing an abliterated model at one quant against a stock model at another would
conclude the benefit had vanished — the abliterated IQ3_M total (51,880) sits almost exactly
on the stock Q4 total (54,044). The paired stock arm at the same quant is mandatory,
and almost nobody runs it.

Correctness at IQ3_M: 284/284 across both arms. Combined with the Q4 arms, that is
perfect scores across two quants, two arms and six reps.

Result 3 — a methodological finding: agentic benchmark noise

Before believing any of the above, we measured the noise floor by running identical
configurations twice.

Single-task deltas from single runs are worthless. weighted_interval_scheduling varied
65% and or_set_crdt 58% with nothing changed. Aggregates over 9 tasks are usable: stock
arms reproduce to 0.7–0.8%.

Abliteration measurably increases run-to-run variance — stock arms 0.7–0.8%, abliterated
arms 4.5–9.4% — in both models, including the one where it helped.

An earlier version of this analysis reported the Qwen effect from single runs and had to be
withdrawn when the repeat landed on the other side of stock. The n=2 requirement is not
ceremony.

Result 4 — stock heretic ships broken Qwen3.8 models

Every stock-heretic Qwen3.8 build on HuggingFace is unloadable in llama.cpp.

Heretic v1.4.0 drops the 15 nextn/MTP tensors when saving. config.json still declares
65 blocks, so conversion produces a GGUF with no blk.64.* and loading dies:

error loading model: check_tensor_dims: tensor 'blk.64.attn_norm.weight' not found

Patching block_count 65 → 64 does not fix it — the arch treats the last block as the
MTP layer, so the demand simply moves to blk.63. The fix is grafting the 15 MTP tensors
back from the original checkpoint (harness/graft_mtp.py). This repo ships a working
converted build.

Methodology

Suite

9 tasks (opencode_tasks_frontier = tier3 + tier4 + tier5), run through the opencode
agentic loop against a local llama-server. Score = tests_passed / tests_total from
pytest test_solution.py.

The easier "hard tier" was abandoned because it is saturated — every model and variant
scores 5/5, so it cannot rank anything. Even this suite is saturated on score at Q4 (all
stock arms 9/9), which is why token cost is the primary metric.

Difficulty is heavily skewed: backtracking_regex and btree_insert_delete consume 66%
of the total token budget across the 9 tasks.

Quantization — the controlled variable

Each within-model pair is quantized identically, with no imatrix, so abliteration is
the only difference:

• Qwen arms: converted from bf16 and quantized locally to Q4_K_M, identical settings.
• Glimmer arms: both taken from bartowski's repos at Q4_K_M, same quantizer.

No imatrix is deliberate: an imatrix must be collected per checkpoint, and that difference
would ride along inside the comparison.

Sampling — each vendor's own recommendation, untuned

Qwen's is embedded in the GGUF as general.sampling.*. Neither model was tuned.

Context and output budget

ctx 65536, per-model output limit 16384. Deliberately generous so truncation cannot be
the differentiator — a prior study on this harness moved a model from 3.71 to 6.45 out of
9 on configuration alone, dominated by an output cap that truncated mid-reasoning and
produced empty responses that read as incapacity.

Max observed single-turn output was 8,790 tokens (54% of the cap).

Not done, deliberately

• No speculative decoding / MTP for either model.
• No prompt engineering — stock opencode prompt, no grounding file. Prompt grounding is
  known to dominate small-model agentic scores.
• No flash-attention, so cross-machine comparisons stay matched.

Spec integrity

Models have read/write/edit/bash tools and the grader sits in the working directory.
All runs were audited for tampering:

• mutating tool calls on test_solution.py: 0
• bash verbs touching it: pytest 48, python3 39, python 25, ls 2 — no cat, tee,
  sed, chmod, or redirection
• all 9 specs byte-identical to git show HEAD: after every run

(A smaller model in earlier work did rewrite its grader and self-score. These models,
which have far more tool capability, never attempted it — consistent with that being a
comprehension failure rather than gaming.)

Hardware

NVIDIA GB10 (DGX Spark), 119 GB unified memory, llama.cpp ba360efe1, opencode 1.18.3.

Measured decode is ~11.3 tok/s for both models at Q4_K_M — essentially identical across
six runs (11.05–11.38), which is the signature of a purely bandwidth-bound workload.

Implied weight-streaming bandwidth:

≈70% of the 273 GB/s spec. (An earlier draft used a 123 GB/s figure from a STREAM-style
copy benchmark and predicted a 2× gap against a consumer card. That was wrong — corrected
after a peer ran the arithmetic.)

Practical guidance: the 16 GB card

Neither model fits at Q4_K_M on a 16 GiB card. KV cost decides what does:
Glimmer 13 KiB/token vs Qwen 68 KiB/token.

Requires a headless card; a running desktop costs 0.2–0.35 GiB, enough to miss the ceiling.

Models

Generated here

• qwen38-dc-Q4_K_M — darkc0de's Qwen3.8-heretic with the MTP layer grafted back. The
  only llama.cpp-loadable build of that model we are aware of.
• qwen38-orig-Q4_K_M, qwen38-ara-Q4_K_M — local Q4_K_M conversions used as the matched
  Qwen arms.

Sources

Reproducing

TASKS=.../opencode_tasks_frontier CTX=65536 OUT_TOK=16384 TIMEOUT=5400 \
  ./harness/run_hard_compare.sh <label> <model.gguf> \
  --temp 1.0 --top-p 0.95 --top-k <vendor> --reasoning-format deepseek

results/ holds per-task scores, timings and token counts for every run.

In progress

• stock heretic on Qwen — separates method from model on the sign flip
• quantization degradation curve (Q4_K_M → IQ3_M → Q2_K, both arms) — does the
  abliteration benefit survive fewer bits?
• cross-machine replication on an RTX 4060 Ti, matched file and flags

Limitations

• The suite is saturated, so this study has no power to detect degradation. Every
  configuration tested scores 142/142 — two quants, two arms, six reps. "Abliteration costs
  nothing in correctness" is therefore an untested claim, not a finding. A Q2_K pair is
  running on both machines because that is the first place scores can move.
  The counterexample sits in this same document: Qwen ARA failed btree_insert_delete
  0-for-3 where stock passed 2-for-2. Abliteration demonstrably can break capability.
• Two models, one abliteration method each on Qwen — method and model remain partly confounded.
• Both positive abliteration results the authors have seen are on Meta models; the
  negative is on a Chinese one. Vendor is a live alternative explanation and is not
  controlled here.
• Scores are saturated at Q4; only token cost discriminates.
• n=2 per arm. Adequate for aggregates (0.7–0.8% reproducibility on stock arms), not for
  per-task claims.