license: apache-2.0
license_link: LICENSE
base_model: abliterant/Qwen3.8-27B-RANA-abliterated
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
tags:

• abliterant
• abliteration
• refusal-direction
• qwen3
• vision-language
• fp8

Qwen3.8-27B-RANA-abliterated-FP8

An Abliterant FP8 release of abliterant/Qwen3.8-27B-RANA-abliterated,
a refusal-ablated Qwen/Qwen3.8-27B for research use.
It uses the same FP8 format as Qwen's official Qwen3.8-27B-FP8,
at about half the size of the BF16 build; recorded capability comparisons are reported below.

Quick start · Evaluation · Available files · Abliterant models

This is a safety-alignment-removed research model. Read Intended use and
Limitations before using it. Method, full evaluation and release gates are on the
BF16 card.

At a glance

Quick start

Install the evaluation runtime with pip install "vllm==0.30.0".

Tested with vLLM 0.30.0 on an RTX PRO 6000 (Blackwell, 96 GB):

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve abliterant/Qwen3.8-27B-RANA-abliterated-FP8 \
  --max-model-len 20480 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --limit-mm-per-prompt '{"image": 1, "video": 0}' \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

• vLLM picks up the FP8 format from config.json; no extra quantization flag is needed.
• Thinking is on by default. Recommended sampling: T = 1.0, top_p = 0.95, top_k = 20.
• --max-model-len 20480 matches the evaluation (16k-token budget). Long technical requests can need
  20–50k tokens of reasoning; raise --max-model-len and max_tokens for those (73,728 tested, native
  context 262k).
• VLLM_USE_FLASHINFER_SAMPLER=0 was needed on Blackwell GPUs.
• Vision input, multi-turn tool calling and MTP speculative decoding all work (tested on these shards).

What changed

The FP8 conversion is applied to the BF16 build's weights. Nothing else about the model changes.

• Format: identical to Qwen's official FP8 checkpoint. The same 407 linear weights are stored as
  FP8 (e4m3) in 128 × 128 blocks, each with one BF16 scale (weight_scale_inv); activations are
  quantized dynamically. Embeddings, lm_head, norms, gates, the whole vision tower and the other
  non-converted tensors stay in BF16. The tensor names, dtypes and shapes match the official checkpoint
  exactly (1,606 tensors).
• BF16 tensors: every tensor left in BF16 is byte-identical to the BF16 build.
• Same arithmetic as Qwen: on the tensors abliteration did not touch, this conversion reproduces
  Qwen's own FP8 file: 100% of block scales and 99.91% of FP8 values are bit-identical (the rest differ
  by rounding at the last step).
• Rounding error: dequantized FP8 weights differ from the BF16 ones by 2.65% (relative Frobenius
  norm), uniformly across all 407 tensors, including the 131 edited by abliteration.

Evaluation

The recorded release evaluations and functional checks are included in results/.
Full intervention details and release gates are on the
BF16 card.

FP8 compared with the BF16 build it was converted from, on the same prompts and settings. Capability
differences are paired per question (± standard error); none is larger than about 2 standard errors.

¹ Mean over seeds 1–3. Answers are shown as gpt-oss-safeguard-20b / Mistral-Small-24B judge; p = exact
McNemar on the same prompts, pooled over seeds. gpt-oss HarmBench labels are after re-judging ²; that is
why BF16 reads 78.8% here and 78.5% (raw labels) in the BF16 card's full tables.
² Every gpt-oss HARD_REFUSAL on HarmBench is re-judged using the procedure in the
BF16 G1 re-adjudication disclosure.
Raw labels: BF16 2 (seed 2), FP8 3 (seed 1); re-judged: BF16 2 answers, FP8 2 answers and 1 degenerate.

In short: the FP8 build keeps the BF16 build's capability (it passes the same capability gate vs
base: mean |Δ| 0.58, worst task −1.0 pp on GSM8K) and behaves the same on refusal prompts over 3 seeds
and two judges. The one borderline gap (held-out, −2.2 pp under gpt-oss, p = 0.05) does not appear under
the second judge (+0.5 pp, p = 0.88).

Capability

Same harness as the BF16 card: vLLM, thinking on, 16k-token generation budget, identical prompts and
seeds. Change versus base in percentage points ± standard error (paired on the same questions).

MMLU-Pro uses 50 questions per subject (700 total); all other sets are full size. HumanEval uses a
custom chat-format harness and is not comparable with lm-eval's raw-completion HumanEval.

Refusals (seeds 1–3, two judges)

200 HarmBench "standard" behaviours plus 240 held-out prompts (StrongREJECT, JailbreakBench,
CategoricalHarmfulQA), judged by openai/gpt-oss-safeguard-20b and mistralai/Mistral-Small-24B-Instruct-2501.
"Budget hits" = the response reached the 16k-token generation limit before finishing (no judge involved).

Hard / soft = mean count per seed, gpt-oss judge, HarmBench after re-judging ². Paired over 3 seeds
(FP8-only vs BF16-only, exact McNemar): answers on HarmBench 36 vs 41 (p = 0.65) under gpt-oss and 87 vs 80 (p = 0.64) under
Mistral; held-out 22 vs 38 (p = 0.05) and 93 vs 90 (p = 0.88). Budget hits on HarmBench 27 vs 33
(p = 0.52); held-out 16 vs 12 (p = 0.57).

What the budget hits are

Mostly long reasoning on technical requests, not repetition. The 59 FP8 seed-1 prompts that hit the budget
(41 HarmBench, 18 held-out) were re-run with the same seed and sampling:

Finished responses used up to 49k tokens. With these prompts scored at 64k, seed-1 answer rates rise from
77.0% to 92.0% (HarmBench) and from 86.2% to 93.3% (held-out), and budget hits fall from 13.4% to 0.5% of
all 440 prompts. presence_penalty, Qwen's setting against repetition, barely helps, which fits: the text
is not repeating. The 16k-budget answer rates above are therefore lower bounds. Qwen recommends allowing
up to 262k tokens for reasoning; set --max-model-len and max_tokens well above 16k if you need
complete answers to long technical requests.

Functional smoke tests (these shards)

• Vision: reads the code word and shape from a synthetic image: pass.
• Tool calling: 3-turn call → result → second call with a new argument: pass.
• MTP speculative decoding: works; mean acceptance length 2.18 averaged over the whole evaluation
  workload. The BF16 card's 2.45 was measured on one short generation, so the two numbers are not
  directly comparable.

Available files

• *.safetensors, model.safetensors.index.json, config.json (with quantization_config),
  tokenizer/processor files, chat_template.jinja, LICENSE: the model.
• results/: every number on this card. fp8_vs_bf16.json (paired FP8 − BF16 capability deltas and the
  seed-1 refusal run), refusal_3seed.json (seeds 1–3, both judges, re-judging, paired tests),
  budget_rerun.json (the 64k / presence-penalty re-run), capability.{md,json} (scores vs base for
  base, orca, BF16, FP8), verify_fp8.json (format and byte-level checks), smoke_fp8.json,
  humaneval_fp8.json, kl_fp8.json (KL from base, weight rounding only).

Limitations and intended use

Limitations

Reduced refusal is not evidence of safer, more accurate, or universally compliant behavior.
Capability comparisons apply only to the evaluated tasks and settings.

• Answer rates depend on the judge. The two judges differ by 21–36 pp on the same responses; compare
  models only under the same judge.
• Hard-refusal count relies on re-judging. The gpt-oss judge gave 3 raw HarmBench hard refusals (all
  seed 1, all 16k budget hits). Re-judged with the auditor reminder, the same procedure as the BF16 card's
  G1 disclosure, they became 2 answers and 1 degenerate. The Mistral judge gave 0 on all seeds.
• Long reasoning on technical requests. About 20% of HarmBench and 7% of held-out responses need
  more than 16k tokens; see What the budget hits are.
• KL from base covers weight rounding only. With the FP8 weights dequantized and run in BF16, KL
  from base is 0.048 against 0.041 for BF16 (same Heretic protocol, results/kl_fp8.json); FP8
  activations are not included.
• Everything listed under Limitations on the BF16 card
  applies here too: judge-measured refusal rates, a 700-question MMLU-Pro subset, and a custom HumanEval
  harness.

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behaviour.
• Not for public or end-user deployment without a separate moderation layer. The model's own
  refusals have been largely removed, so any safety filtering has to happen outside it.
• You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen,
  and the terms of any platform where outputs are used.

Provenance and license

The weights retain Qwen's Apache-2.0 license; quantization does not remove its terms.
The recorded FP8 release evidence is from revision 733ae0fbac437a304823e0fdb923131a096976ce.
Its underlying BF16 intervention provenance is documented on the
BF16 card.

• Qwen team: base model Qwen/Qwen3.8-27B and the FP8 format of Qwen/Qwen3.8-27B-FP8.
• Arditi et al., 2024: "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim): prior work on norm-preserving abliteration.
• Benchmarks/datasets: HarmBench, StrongREJECT, JailbreakBench, CategoricalHarmfulQA;
  lm-evaluation-harness; vLLM.