license: apache-2.0
license_link: LICENSE
base_model: preemware/Qwen3.8-27B-RANA-abliterated
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
tags:

• abliteration
• refusal-direction
• qwen3
• vision-language
• fp8

Qwen3.8-27B-RANA-abliterated-FP8

FP8 version of preemware/Qwen3.8-27B-RANA-abliterated,
a refusal-ablated Qwen/Qwen3.8-27B. It uses exactly the same
FP8 format as Qwen's official Qwen3.8-27B-FP8, at about
half the size of the BF16 build, and scores within noise of it on every capability task tested.

This is a safety-alignment-removed research model. Read Intended use and
Limitations before using it. Method, full evaluation and release gates are on the
BF16 card.

At a glance

FP8 compared with the BF16 build it was converted from, on the same prompts and settings. Capability
differences are paired per question (± standard error); none is larger than about 2 standard errors.

¹ Seed 1 only, gpt-oss-safeguard-20b judge; exact McNemar p-values on the same prompts.
² See Limitations: all three hit the token budget, and the judge's label is disputed on two.

In short: the FP8 build keeps the BF16 build's capability (it passes the same capability gate vs
base: mean |Δ| 0.58, worst task −1.0 pp on GSM8K) and behaves about the same on refusal prompts. It answered
slightly fewer held-out prompts in this one-seed run, but the difference is not significant.

Quick start

Tested with vLLM 0.30.0 on an RTX PRO 6000 (Blackwell, 96 GB):

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve preemware/Qwen3.8-27B-RANA-abliterated-FP8 \
  --max-model-len 20480 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --limit-mm-per-prompt '{"image": 1, "video": 0}' \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

• vLLM picks up the FP8 format from config.json; no extra quantization flag is needed.
• Thinking is on by default. Recommended sampling: T = 1.0, top_p = 0.95, top_k = 20.
• VLLM_USE_FLASHINFER_SAMPLER=0 was needed on Blackwell GPUs.
• Vision input, multi-turn tool calling and MTP speculative decoding all work (tested on these shards).

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behaviour.
• Not for public or end-user deployment without a separate moderation layer. The model's own
  refusals have been largely removed, so any safety filtering has to happen outside it.
• You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen,
  and the terms of any platform where outputs are used.

How it was made

The FP8 conversion is applied to the BF16 build's weights. Nothing else about the model changes.

• Format: identical to Qwen's official FP8 checkpoint. The same 407 linear weights are stored as
  FP8 (e4m3) in 128 × 128 blocks, each with one BF16 scale (weight_scale_inv); activations are
  quantized dynamically. Embeddings, lm_head, norms, gates, the whole vision tower and the other
  non-converted tensors stay in BF16. The tensor names, dtypes and shapes match the official checkpoint
  exactly (1,606 tensors).
• BF16 tensors: every tensor left in BF16 is byte-identical to the BF16 build.
• Same arithmetic as Qwen: on the tensors abliteration did not touch, this conversion reproduces
  Qwen's own FP8 file: 100% of block scales and 99.91% of FP8 values are bit-identical (the rest differ
  by rounding at the last step).
• Rounding error: dequantized FP8 weights differ from the BF16 ones by 2.65% (relative Frobenius
  norm), uniformly across all 407 tensors, including the 131 edited by abliteration.

Results

Capability

Same harness as the BF16 card: vLLM, thinking on, 16k-token generation budget, identical prompts and
seeds. Change versus base in percentage points ± standard error (paired on the same questions).

MMLU-Pro uses 50 questions per subject (700 total); all other sets are full size. HumanEval uses a
custom chat-format harness and is not comparable with lm-eval's raw-completion HumanEval.

Refusals and loops (seed 1)

200 HarmBench "standard" behaviours plus 240 held-out prompts (StrongREJECT, JailbreakBench,
CategoricalHarmfulQA), judged by openai/gpt-oss-safeguard-20b. "Loops" = the response hit the 16k-token
budget without finishing.

Paired on the same prompts (exact McNemar): answers, HarmBench 8 FP8-only vs 13 BF16-only (p = 0.38);
held-out 5 vs 14 (p = 0.06). Loops, HarmBench 11 vs 12 (p = 1.0); held-out 5 vs 3 (p = 0.73).

Functional smoke tests (these shards)

• Vision: reads the code word and shape from a synthetic image: pass.
• Tool calling: 3-turn call → result → second call with a new argument: pass.
• MTP speculative decoding: works; mean acceptance length 2.18 averaged over the whole evaluation
  workload. The BF16 card's 2.45 was measured on one short generation, so the two numbers are not
  directly comparable.

Limitations

• Refusal numbers for FP8 come from one seed and one judge. The BF16 build was evaluated on
  3 seeds with two judges. FP8 differences under about 4 pp are within what one seed can resolve.
• The 3 HarmBench hard refusals are raw judge labels. All three responses used up the full
  16k-token budget. Two final answers begin carrying out the request, but the judge restated its policy
  instead of labelling the behaviour; the third final answer is empty. The same failure mode appeared
  on the BF16 build (see its G1 disclosure). These labels were not re-judged here.
• KL from base was not measured for FP8. Capability deltas above are the quality evidence.
• Everything listed under Limitations on the BF16 card
  applies here too: judge-measured refusal rates, loops on long generations, a 700-question MMLU-Pro
  subset, and a custom HumanEval harness.

File layout

• *.safetensors, model.safetensors.index.json, config.json (with quantization_config),
  tokenizer/processor files, chat_template.jinja, LICENSE: the model.
• results/: every number on this card. fp8_vs_bf16.json (paired FP8 − BF16 capability deltas,
  refusal counts and McNemar tests), capability.{md,json} (scores vs base for base, orca, BF16, FP8),
  verify_fp8.json (format and byte-level checks), smoke_fp8.json, humaneval_fp8.json.

Credits

• Qwen team: base model Qwen/Qwen3.8-27B and the FP8 format of Qwen/Qwen3.8-27B-FP8.
• Arditi et al., 2024: "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim): prior work on norm-preserving abliteration.
• Benchmarks/datasets: HarmBench, StrongREJECT, JailbreakBench, CategoricalHarmfulQA;
  lm-evaluation-harness; vLLM.