license: apache-2.0
base_model: Qwen/Qwen3-235B-A22B
tags:

• abliterated
• uncensored
• qwen3
• moe
• ablation
• fp8
• quantized
• compressed-tensors
  language:
• en
• zh
• ja
• ko
• fr
• de
• es
• pt
• ru
• ar
• th
• vi
  library_name: transformers
  pipeline_tag: text-generation
  model-index:
• name: Qwen3-235B-A22B-abliterated-FP8
  results: []

Qwen3-235B-A22B-abliterated-FP8

An FP8-quantized abliterated version of Qwen/Qwen3-235B-A22B. This is the recommended version for inference — it fits on 4x RTX Pro 6000 GPUs (384 GB total) and serves well under vLLM.

Abliteration removes the dominant refusal direction from model weights using the technique from Refusal in Language Models Is Mediated by a Single Direction (Arditi et al.), making the model significantly less likely to refuse prompts while retaining its full capabilities.

This started as research into abliteration, but also as a search for the best creative writing model I could run locally. Qwen3-235B has excellent prose quality, but its refusal behavior gets in the way of fiction — injecting disclaimers, refusing to write morally complex characters, hedging on anything edgy. Abliteration fixes this well, especially with a good system prompt. The BF16 weights (~438 GB) don't fit in 384 GB of VRAM, so this FP8 version is what I actually serve.

The full-precision BF16 version is also available: null-space/Qwen3-235B-A22B-abliterated

A vision-language variant is also available: null-space/Qwen3-VL-235B-A22B-Abliterated-FP8

Benchmarks

MMLU (5-shot)

Evaluated using lm-evaluation-harness v0.4.11 against the vLLM-served FP8 model. Baseline is the published score for Qwen3-235B-A22B (source).

The 1.6% drop is within the acceptable range for abliteration (<2%), indicating the technique preserved the model's general knowledge and reasoning capabilities.

Quantization Details

This quantization halves the storage from the BF16 version (~438 GB to ~221 GB) while maintaining near-lossless quality. Compatible with vLLM and other frameworks supporting the compressed-tensors format.

How It Was Made

1. Abliteration was performed on the BF16 base model — refusal directions measured across all 94 layers were projected out of o_proj and down_proj weight matrices for layers 21-93, with variable per-layer scale factors (0.3-1.0).
2. FP8 quantization was then applied to the abliterated BF16 weights using block-wise static quantization (compressed-tensors format).

See the BF16 model card for full ablation configuration details.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "null-space/Qwen3-235B-A22B-abliterated-FP8"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Your prompt here"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Recommended Serving

For serving this FP8 model, vLLM with tensor parallelism is recommended:

vllm serve null-space/Qwen3-235B-A22B-abliterated-FP8 \
    --tensor-parallel-size 2 \
    --max-model-len 8192

The FP8 version can serve with fewer GPUs than the BF16 version thanks to its reduced memory footprint.

Model Details

Ethical Notice

This model has had its refusal training removed. It will comply with requests that the original model would refuse. You are solely responsible for how you use this model. It is intended for research into LLM alignment, safety evaluation, red-teaming, and understanding refusal mechanisms.

Credits

• Base model: Qwen Team
• Abliteration technique: Based on Refusal in Language Models Is Mediated by a Single Direction by Arditi et al.