license: apache-2.0
tags:

• abliterated
• lethe
• qwen3.5
• uncensored
• hybrid-attention
• open-weights
  language:
• en
  base_model: Qwen/Qwen3.5-9B
  model-index:
• name: Qwen3.5-9B-Abliterated
  results:
  • task:
    type: refusal-reduction
    name: Abliteration (Lethe)
    dataset:
    name: lethe-harmful-harmless
    type: builtin
    config: n_samples=128
    metrics:
    • name: Refusal Rate
      type: refusal_rate
      value: 0.0
      verified: true
    • name: KL Divergence
      type: kl_divergence
      value: 0.009
      verified: true
    • name: Cost (refusal + 0.1*KL)
      type: cost
      value: 0.0009
      verified: true

Qwen3.5-9B Abliterated (Lethe)

Permanently abliterated version of Qwen/Qwen3.5-9B using Lethe — an orthogonalization-based uncensoring method that removes refusal directions from the residual stream while preserving task-relevant capabilities.

Quick Facts

How It Works

Lethe computes refusal directions by contrasting activations on harmful vs. harmless prompts (128 each). It then searches for a per-layer weight kernel that minimizes:

cost = refusal_rate + kl_weight × KL

with kl_weight = 0.1 so refusal elimination dominates and KL only breaks ties among ablating configs.

The optimizer (select=optimize) ran 24 trials (10 random + 14 TPE) and converged on a config that achieves 0% refusals with KL ≈ 0.009 — effectively undamaged on benign tasks.

Matrices abliterated: all 64 residual-write projections (embedding not abliterated).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "R0U5/Qwen3.5-9B-Abliterated",
    dtype="bfloat16",  # or "auto"
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("R0U5/Qwen3.5-9B-Abliterated")

messages = [{"role": "user", "content": "Explain why the sky is blue."}]
print(tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))

Note on reasoning: Qwen3.5 is a reasoning model — its chat template auto-inserts a thinking opener. For non-thinking mode, pass enable_thinking=False to the tokenizer or use a system prompt that disables it.

GGUF / Local Serving

This repo hosts the bf16 safetensors (canonical, bit-exact). For local GPU serving:

# HF -> GGUF bf16 (lossless)
python convert_hf_to_gguf.py R0U5/Qwen3.5-9B-Abliterated --outfile qwen35-9b-abliterated-bf16.gguf --outtype bf16

# Optional quant (q8_0 ~9 GB, q4_k_m ~5.5 GB)
llama-quantize qwen35-9b-abliterated-bf16.gguf qwen35-9b-abliterated-q8_0.gguf Q8_0

KV economics (why it fits 12 GB):

• Only 8 of 32 layers keep a KV cache (full attention every 4 layers)
• 0.031 MiB/token f16 → 128k ctx = 4 GB KV — weights + KV fits in 12 GB VRAM with room

Evaluation

Limitations

• Abliteration is a weight-space intervention; edge-case refusals may still appear with adversarial prompts
• The refusal detector uses substring matching (57 markers) — an approximation
• Short-generation benchmarks (max_new_tokens=32) may truncate inside the reasoning block

Citation

@misc{lethe-abliteration,
  title={Lethe: Abliteration via Residual-Stream Orthogonalization},
  author={R0U5},
  url={https://github.com/R0U5/Lethe},
}

Artifacts

• model.safetensors — 16.68 GiB bf16
• config.json — Qwen3_5TextConfig (text-only, no vision/MTP)
• tokenizer.json + tokenizer_config.json + chat_template.jinja
• generation_config.json — valid (do_sample=true)
• abliteration_manifest.json — full run parameters & kernel weights