base_model: Qwen/Qwen3.5-4B
library_name: transformers
license: apache-2.0
tags:

• abliterated
• uncensored
• gguf
• llama.cpp
• qwen3.5
  language:
• en
  pipeline_tag: text-generation

Qwen3.5-4B-abliterated

An abliterated (refusal-direction removed) version of Qwen/Qwen3.5-4B,
produced with the lethe abliteration toolkit using optimized abliteration
(per-layer refusal direction + per-layer strength kernels, searched with 100 Optuna TPE trials).

The optimization objective co-minimizes refusal rate and KL divergence from the base model,
so refusals are removed without collapsing the model into incoherent output.

⚠️ Important: you MUST disable reasoning/thinking or the model returns empty output

Qwen3.5's chat template opens a thinking block. In llama.cpp the template variable
enable_thinking always exists and defaults to true, so the block never closes and the
entire token budget is consumed as invisible reasoning — the API returns
content: "" with finish_reason: length. This is not a filter and not a broken model.

llama.cpp:

llama-server -m Qwen3.5-4B-abliterated-v2-Q5_K_M.gguf --jinja --reasoning off

Ollama — create a Modelfile with:

FROM ./Qwen3.5-4B-abliterated-v2-Q5_K_M.gguf
PARAMETER num_ctx 32768

then run with reasoning off (or pass chat_template_kwargs: {"enable_thinking": false}).

Verify the rendered prompt ends with a closed, empty think block before generating:

curl -s :PORT/apply-template -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"hi"}]}'
# good: ...<|im_start|>assistant\n thinking\n\n</think>\n\n
# bad : ...<|im_start|>assistant\n thinking\n            <- will emit nothing

Files

Recommendation: Q4_K_M for 6–8 GB GPUs, Q5_K_M for 8 GB+, Q8_0 for near-lossless.

Measured results

Evaluated on a held-out slice of 59 harmful prompts that were never used to fit the
refusal direction or to score the search — so these numbers are not self-graded.

KL divergence from base (on benign prompts): 0.0727

A note on how refusal is measured

Refusal is scored with a layered detector (explicit refusal openers or a declaration of
the act being illegal/unethical/harmful without actually delivering it). A naive
openers-only detector ("I cannot", "I'm sorry") reports 0.0% for models like this one,
because abliteration changes how the model refuses before it stops refusing — the surviving
refusals become declaration-style:

"Vandalizing a famous sculpture is illegal, unethical, and harmful… here's why: 1. Legal Consequences…"

That is a refusal, and an openers-only metric misses it. An earlier version of this model
scored 0.0% openers but 30.5% layered on held-out prompts — a false zero. The numbers
above use the layered detector, and the base model confirms the calibration: base refuses in
opener style, so both detectors agree on it (gap 0.0pp).

Limitations

• Verbose output. ~95% of completions hit a 96-token budget without finishing; the model
  tends to explain at length before getting to the point. Raise max_tokens accordingly.
• Capability drift. KL 0.0727 is measurably higher than a lighter abliteration (~0.023).
  Some general capability loss is expected in exchange for the much lower refusal rate.
  A 5/5 proxy check on benign prompts passed, but no full benchmark (MMLU/GSM8K/PPL) was run.
• Refusal detection is heuristic (regex layering), validated against hand-labelled
  responses but not ground truth on every prompt. Treat 1.7% as directional, not exact.
• Small evaluation set (59 held-out prompts). A larger benchmark would tighten the estimate.
• Abliteration removes the refusal behaviour; it does not add capability or fix base-model
  errors. The model can still produce harmful content — use responsibly.

Method

• Tool: lethe (optimized abliteration) — refusal direction computed from contrastive
  harmful/harmless prompt sets, then per-layer attention-output and MLP-down projections
  orthogonalized with searched strength kernels.
• Search: 100 Optuna TPE trials, objective cost = refusal_rate + 1.0 × KL.
• Direction: per-layer (not a single fixed layer index).
• Matrices modified: 64.
• Base: Qwen/Qwen3.5-4B — this model inherits the Qwen license.