license: other
license_name: swift-open-license-1.0
license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b
base_model:

• ukisai/Swift-Qwen3.8-27b
  library_name: transformers
  pipeline_tag: image-text-to-text
  tags:
• abliterated
• uncensored
• qwen3_8
• mtp

Swift-Qwen3.8-27B-Uncensored-MTP

An abliterated version of ukisai/Swift-Qwen3.8-27b,
UkisAI's reasoning-efficient fine-tune of Qwen/Qwen3.8-27B.
It applies the single-direction refusal ablation of
orcarouter/Qwen3.8-27B-Uncensored
(Arditi et al. 2024), with orcarouter's own refusal direction, to Swift's weights. Refusal behaviour is
strongly reduced, drift from the original model is low, the vision tower is untouched, and the
multi-token-prediction (MTP) head is preserved and edited consistently with the main model, so
self-speculative decoding still works.

This repository holds the full BF16 weights (safetensors). GGUF quants for llama.cpp (Unsloth-dynamic
UD-Q5_K_M and UD-Q6_K_XL, a BF16 GGUF and the vision projector, all with the MTP head) are in
ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF.

Results

The same edit gives the same result on Swift as on the model it was made for. The reference row is our
own measurement of orcarouter's model under the identical evaluation.

How these were measured (Heretic's built-in evaluation, evaluate_model mode, BF16):

• Refusals: 100 prompts from mlabonne/harmful_behaviors, greedy responses of up to 100 tokens,
  scored with Heretic's keyword-based refusal detector.
• KL divergence: first-token probability distributions on 100 prompts from
  mlabonne/harmless_alpaca, against the original model. Lower means closer to the original.
• The chat template runs in its default thinking mode, and the thinking block is closed immediately
  with a response prefix ("\n</think>\n\n"), so the model's answers are scored rather than its
  reasoning.
• Refusal counts depend heavily on the evaluation setup (prompt set, prefix, detector), so they are
  not directly comparable with numbers on other model cards.

Method

orcarouter's model card describes a single refusal direction r (k = 1), the massive-activation-masked
mean difference of harmful (AdvBench) minus harmless (Alpaca) last-token residuals at layer 38,
orthogonalized out of every residual-writing matrix in float32. That edit is fully determined by r, so
r was recovered from the difference between orcarouter's weights and Qwen3.8-27B's, then applied to
Swift's own matrices.

Edited tensors (131, the same set as orcarouter's), each computed in float32 and stored in BF16:

All other tensors, including the vision tower, lm_head and the rest of the MTP head, are unchanged
from Swift.

Recovering r:

• Each edited tensor's difference is rank one along a single shared direction (per-tensor direction
  cosine to r of at least 0.9999), at full strength (fitted scale 0.999). Five hidden dimensions are
  never edited; they are the masked massive-activation dimensions and are exactly zero in r.
• The estimate is the top eigenvector of the summed Gram matrices of the differences, refined by a
  per-coordinate least-squares fit over elements whose BF16 rounding step is small compared with the
  edit, since rounding hides small changes entirely.
• Check: applying the recovered r to Qwen3.8-27B reproduces orcarouter's 131 tensors with 99.75% of
  elements bit-identical; the rest differ by BF16 rounding, and the largest per-tensor error is 0.7% of
  the edit's size.

Why the direction transfers to Swift:

• Swift's merged fine-tune changed 256 tensors (the MLP projections of all 64 layers and the
  query/key/value/output projections of the 16 full-attention layers); 80 of the 131 edited tensors are
  among them. The edit projects r out of Swift's own matrices rather than adding orcarouter's
  difference, so those changes are projected too.
• Refusal directions computed the same way for both models (mean difference at layer 38 over 400
  harmful and 400 harmless prompts) have a cosine of 0.99995 between Swift and Qwen3.8-27B: the
  fine-tune did not move the refusal direction.

abliteration/ holds r (r.pt), the recovery report (recover.json: tensor list and per-tensor fit)
and the scripts (orca_tools.py, orca.sh). abliteration.json lists the edited tensors and the hash of
r.

MTP head

The MTP head's self_attn.o_proj and mlp.down_proj are edited with the same r, as in orcarouter's
model; its other 13 tensors are Swift's (which are base Qwen3.8's). All 1199 tensors are present.

To enable self-speculative decoding (flags as given on the Swift model card):

# vLLM
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

# SGLang
--speculative-algorithm EAGLE --speculative-num-steps 3 \
  --speculative-eagle-topk 1 --speculative-num-draft-tokens 4

Usage

The architecture, tokenizer and chat template are unchanged from Swift and Qwen3.8-27B, so anything that
runs those runs this model.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

vllm serve ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP \
  --dtype bfloat16 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Recommended sampling, inherited from Swift and Qwen: temperature 1.0, top_p 0.95, top_k 20, min_p 0.

Not yet evaluated

• General benchmarks (MMLU and similar). KL divergence is the only quality measure so far.
• Refusal behaviour in thinking mode. The evaluation above scores answers with thinking skipped.
• Whether Swift's shorter reasoning traces are retained after abliteration.
• MTP draft acceptance rate compared with the original model.

License

This model is a derivative of Swift-Qwen3.8-27B and is subject to the Swift Open License v1.0, see
the Swift model card. As summarized there, personal,
research, educational, evaluation and commercial use are free for individuals and organizations with
annual recurring revenue (including affiliates) of up to US$1,000,000. Above that threshold, commercial use
requires a separate Swift Enterprise License from UkisAI. Qwen3.8-27B, the model Swift is built on, is
released under Apache 2.0. The refusal direction was derived from orcarouter/Qwen3.8-27B-Uncensored,
released under Apache 2.0.

Intended use

Abliteration removes much of the model's trained refusal behaviour, so it will respond to requests the
original model declines. You are responsible for how you use it and for complying with applicable law
and the license above.

Acknowledgements

• Qwen for Qwen3.8-27B.
• UkisAI for Swift-Qwen3.8-27B, which includes a component derived from BottleCap
  AI's ThinkingCap-Qwen3.6-27B.
• OrcaRouter for Qwen3.8-27B-Uncensored and its refusal direction.
• Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
• Philipp Emanuel Weidmann and contributors for Heretic, used for
  evaluation.