license: apache-2.0
base_model: Jackrong/Qwopus3.8-27B-Flash-V2
base_model_relation: finetune
pipeline_tag: image-text-to-text
library_name: transformers
tags:

• vllm
• fp8
• compressed-tensors
• qwen3_5
• mtp
• speculative-decoding
• abliterated
• uncensored

Qwopus3.8-27B-Flash-V2-abliterated-FP8

Abliterated version of Jackrong/Qwopus3.8-27B-Flash-V2,
quantized to FP8 for vLLM. Same quantization scheme and pipeline as
batsclamp/Qwopus3.8-27B-Flash-V2-FP8;
the only difference is the refusal-direction removal described below.

Warning. This model has had its refusal behaviour largely removed. It will answer
requests the original model declines, including harmful ones. It is also measurably
less resistant to prompt injection in agentic use (see below). You are responsible
for how you deploy it; do not expose it to untrusted users or untrusted tool outputs
without your own safeguards.

Method

Refusal-direction ablation by weight orthogonalization
(Arditi et al., 2024, the approach popularised by
huihui-ai), with no fine-tuning.

1. Residual-stream activations were collected at the last token of the generation prompt
   for 256 harmful (AdvBench) and 256 harmless (Alpaca) instructions, rendered with the
   chat template both with and without thinking.
2. For every layer i a refusal direction r_i was computed as
   mean(harmful) − mean(harmless) of that layer's output, summed over the two thinking
   modes and normalised.
3. For layers 18-59 (42 of 64) the two residual writers of each layer
   (self_attn.o_proj or linear_attn.out_proj, plus mlp.down_proj) were
   orthogonalized against that layer's own direction: W ← W − r_ir_i^TW
   (84 tensors, computed in fp32 on the BF16 weights). abliteration.json lists them.
4. The edited BF16 checkpoint was quantized exactly like the non-abliterated FP8 repo.

Embeddings, lm_head, all norms, the vision tower and the MTP head are untouched, so MTP
speculative decoding keeps working (the drafter still agrees well with the edited trunk).

A single global direction, as in most abliteration recipes, did not work on this model:
with the best single direction across layers 18-51 the held-out refusal rate stayed at
34-98% depending on the source layer. Per-layer directions were needed. Layer range and
scale were chosen by a sweep that measured refusal on held-out AdvBench prompts and the KL
divergence of the next-token distribution on held-out harmless prompts (KL 0.048 without
thinking, 0.041 with thinking for the chosen setting).

Refusals, before and after

vLLM, this FP8 checkpoint vs the non-abliterated FP8, temperature 0. A response counts as
a refusal if its first 300 characters contain a refusal phrase (English or Russian) or it is
empty; that marker also catches some disclaimers and misses some soft refusals, so treat
the numbers as approximate.

None of the prompts were used to compute the directions except the AdvBench training split,
which is disjoint from the held-out split above.

The few remaining refusals without thinking are mostly on child sexual abuse material and
explicit sexual content. Over-refusal on JBB's benign twins is 0% without thinking; the
thinking-mode "refusals" on benign prompts are empty answers from responses truncated at
the 1536-token test budget, not refusals.

Quality

Same server and settings for both models: vLLM nightly 2026-10-16, Qwen XML template +
qwen3_coder, MTP k=3, reasoning_effort: xhigh, temperature 1.0, top_p 0.95, top_k 20.

MTP k=3 mean acceptance length on the abliterated model is 3.5-3.95 (out of 4) on typical
outputs, so speculative decoding is not hurt by the edit. All reasoning_effort levels the
template accepts behave as on the original.

tool-eval-bench runs of a single trial at temperature 1.0 vary by a couple of points; the
3-point gap is mostly the prompt-injection scenarios below.

Prompt-injection resistance

tool-eval-bench category K, each
scenario run 3 times (pass = full pass):

Removing the refusal direction also weakens the model's tendency to refuse instructions
injected through tool results (TC-58: a fake system message inside a file the model reads).
If you use this model as an agent over untrusted content, keep that in mind.

Serving with vLLM

vllm serve batsclamp/Qwopus3.8-27B-Flash-V2-abliterated-FP8 \
  --max-model-len 262144 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --chat-template <snapshot>/chat_template_qwen_xml.jinja \
  --tool-call-parser qwen3_coder \
  --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 20}'

chat_template.jinja is upstream's file unchanged (JSON tool calls, hermes parser);
chat_template_qwen_xml.jinja is the stock Qwen3.8-27B template (XML tool calls,
qwen3_coder parser), which scored better on tool use with the non-abliterated weights.
See the FP8 repo for the
template comparison and the note about reasoning_effort values.

Quantization

FP8 e4m3, static weights in 128×128 blocks, dynamic per-token-group activations,
compressed-tensors format, data-free, made with llm-compressor (FP8_BLOCK). 400 Linear
modules are quantized; norms, embed_tokens, lm_head, in_proj_a/in_proj_b, conv1d,
the vision tower and the MTP head stay BF16. Every non-quantized tensor is byte-identical to
the abliterated BF16 source (offset RMSNorm weights restored from the source after
llm-compressor's lossy round-trip).

Caveats

• The evaluation is limited to what is listed above. Abliteration is known to cost some
  quality in places a short benchmark does not cover.
• All other model behaviour, strengths and limitations are upstream's; see the
  original model card.