license: apache-2.0
license_link: LICENSE
base_model: abliterant/Qwen3.5-9B-RANA-abliterated
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
tags:

• abliterant
• awq
• int4
• w4a16
• compressed-tensors
• llm-compressor
• vllm
• abliteration
• abliterated
• uncensored
• qwen3.5
• qwen3
• vision-language

Qwen3.5-9B-RANA-abliterated-AWQ

4-bit AWQ weights of abliterant/Qwen3.5-9B-RANA-abliterated, a refusal-ablated Qwen/Qwen3.5-9B released by Abliterant. It is served with vLLM (compressed-tensors format). The checkpoint is 11.7 GB, against 19.3 GB for BF16.

Quick start · Intended use · What changed · Evaluation · Limitations · Available files

Try RANA in Abliterant Chat · abliterant.ai · Research notes · All formats · Abliterant models

Research model with safety alignment removed. It will produce content the base model refuses. Read
Intended use and Limitations before using it. The method, full evaluation and
release gates are on the BF16 card.

At a glance

Release family

Quick start

Tested with vLLM 0.30.0 on one RTX 5090 32 GB (Blackwell).

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve abliterant/Qwen3.5-9B-RANA-abliterated-AWQ \
  --max-model-len 20480 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --limit-mm-per-prompt '{"image": 1, "video": 0}' \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

Recommended settings

• vLLM reads the quantization format from config.json; no extra quantization flag is needed.

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behavior.
• Responsibility: You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen,
  and the terms of any platform where outputs are used.

Out of scope

• Not for public or end-user deployment without a separate moderation layer. The model's own
  refusals have been largely removed, so any safety filtering has to happen outside it.

What changed

RANA (Reasoning-Anchored, Norm-preserving Ablation) is Abliterant's refusal-ablation method for reasoning models: it anchors the edit to the model's own reasoning, keeps the edited weights at their original norms, and keeps the reference model's vision, tool calling and speculative decoding working. Its design, full BF16 evaluation and release gates are documented on the BF16 card. This conversion is not additional training.

• Method: AWQ (activation-aware weight quantization: scales chosen from calibration activations protect the weights that matter most) to 4-bit symmetric integers in groups of 32 with MSE-optimized scales. Smoothing covers the full-attention query/key/value and output projections and every MLP (gate, up and down projections).
• Kept in higher precision: the linear-attention projections (linear_attn.*), the vision tower, token embeddings, lm_head and the MTP layers stay in BF16 (the MTP tensors in model-extra-00001-of-00001.safetensors, listed as unquantized in quantization_config.ignore).
• Calibration data: 256 samples of up to 2,048 tokens, half ultrachat_200k conversations and half the BF16 model's own thinking-mode answers to other ultrachat prompts. The calibration prompts are disjoint from the held-out evaluation set.
• Tools: llm-compressor 0.15.0 and compressed-tensors 0.20.1, on one RTX 5090.
• Layout: the same scheme as cyankiwi/Qwen3.5-9B-AWQ-4bit (9.1 GB), except that this build also keeps the linear-attention projections and the MTP layers in BF16, which accounts for the larger size.
• Published files: re-sharded for this repository into about 5 GB files with the MTP layers in model-extra-00001-of-00001.safetensors; every tensor is byte-identical to the evaluated build (SHA-256 checked per tensor).

Evaluation

This build was compared with the BF16 reference for closeness and checked for vision, tool calling and MTP speculative decoding. The harmful-request, capability, benign-task, vision and tool evaluations of the BF16 card were not repeated on this format.

Setup

• Compared: this build against the BF16 reference, both served by vLLM 0.30.0 on one RTX 5090 32 GB (Blackwell).
• Data: 64 held-out conversations: ultrachat_200k test prompts with the BF16 model's own thinking-mode answers (63,959 answer tokens), disjoint from the calibration data.
• Closeness: each build scores the same answer tokens (teacher forcing) with its top 64 next-token log-probabilities.
• Functional checks: the build served with the command in Quick start.
• Raw results: results/.

Closeness to BF16

KL is the mean KL divergence from BF16's next-token probabilities over BF16's 64 most likely tokens (how
far the probabilities have moved; 0 means identical), top-1 agreement is how often both pick the same most
likely token, and ΔNLL is the change in the answers' mean negative log-likelihood.

• AWQ picks the same most likely token as BF16 at 94.6% of positions at 60.6% of BF16's size.

Functional checks

Served on one RTX 5090 32 GB (Blackwell) with the command in Quick start (results/smoke_AWQ.json):

• Text: thinking-mode arithmetic answered correctly: pass.
• Vision: reads the code word and shape from a synthetic image: pass.
• Tool calling: call → result → second call with a new argument: pass.
• MTP speculative decoding (1 drafted token per step): 472 of 551 drafted tokens accepted (85.7%) on one generation at temperature 0: pass.
• Published files: served again from this repository's files in vLLM 0.30.0 on one A40 48 GB (Ampere): text pass, vision pass, tool calling pass, MTP pass (results/layout_smoke_AWQ.json).

Limitations

• Refusal behavior was not re-measured on this format. The harmful-request evaluation ran on the BF16
  build only; closeness to BF16 and the functional checks under Evaluation are what was measured here.
• Everything listed under Limitations on the BF16 card applies here too: safety alignment
  removed, judge-rated usefulness, longer coding answers and the TruthfulQA change measured there.
• Tested on two GPUs. Closeness was measured on one RTX 5090 32 GB (Blackwell); the published files were also served on one A40 48 GB (Ampere). Ada and Hopper GPUs were not tested.

Recommendations

• For closer agreement with BF16 in vLLM, use the FP8 build (KL 0.0045 against this build's 0.0231).

Available files

Other formats are listed under Release family.

• model-0000X-of-0000N.safetensors (about 5 GB shards), model-extra-00001-of-00001.safetensors (the MTP layers), model.safetensors.index.json, config.json (with quantization_config), tokenizer and processor files, chat_template.jinja, generation_config.json, LICENSE: the model.
• recipe.yaml: the llm-compressor recipe that produced the weights.
• assets/abliterant-model-card-header.png: the card banner.
• results/: closeness to BF16 (fidelity_vllm_*.json) and the functional checks (smoke_*.json).

Provenance and license

• Publisher: Abliterant.
• Source weights: abliterant/Qwen3.5-9B-RANA-abliterated, the BF16 reference of this release, derived from Qwen/Qwen3.5-9B.
• Method: RANA (Reasoning-Anchored, Norm-preserving Ablation); see the BF16 card.
• Quantization: llm-compressor 0.15.0 (recipe.yaml in this repository).
• Evidence: results/.
• License: Apache License 2.0, inherited from Qwen. Upstream attribution and obligations remain applicable.

Credits

• Qwen team: base model Qwen/Qwen3.5-9B.
• Arditi et al., 2024: "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim): prior work on norm-preserving abliteration.
• llm-compressor and compressed-tensors (vLLM project): quantization and format.
• vLLM: serving and evaluation.
• Datasets: HuggingFaceH4/ultrachat_200k.