license: other
license_name: swift-open-license-1.0
license_link: LICENSE
gated: true
pipeline_tag: image-text-to-text
tags:

• qwen3_5
• autoround
• w4a16
• uncensored
• abliterated
• reasoning-efficient
• vllm
  base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
  base_model_relation: quantized

Swift-Qwen3.8-27B-Uncensored-W4A16-AutoRound

A W4A16 AutoRound quantization of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16,
reproducing dbirks/Qwen3.8-27B-W4A16-AutoRound's
published recipe against this uncensored, reasoning-efficient base instead
of the official checkpoint. Combines ukisai/Swift-Qwen3.8-27b's
reasoning-token-efficiency adapter with d0xin's abliteration, quantized to
run comfortably on a single 24GB GPU with speculative decoding intact.

Provenance

Qwen/Qwen3.8-27B (Apache 2.0)
  -> ukisai/Swift-Qwen3.8-27b (Swift Open License v1.0)
       reasoning-efficiency LoRA adapter, merged
  -> d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
       rank-1 directional residual-stream ablation, layer 38
       (131 / 1,199 tensors modified, vision tensors unchanged)
  -> this repository
       AutoRound W4A16 g128 sym quantization + int8 head/embed/MTP shrink

See NOTICE for the full attribution chain and license obligations.

Tested with

Quantized and served on a single RTX 3090 (24GB) using
syv-ai/HyperQwen's vLLM 0.28.0
container (DFlash2 MTP speculative decoding, --profile single). Both
context tiers documented there (CTX=fast, 65,536 tokens; CTX=huge,
245,760 tokens via KVarN) boot and serve correctly with this checkpoint --
see the throughput numbers below.

Throughput

Single-stream decode, DFlash2 MTP speculative decoding, RTX 3090, from a
real /v1/chat/completions request (not a synthetic benchmark harness --
we didn't run llama-bench here since that targets GGUF/llama.cpp and this
checkpoint is a vLLM/compressed-tensors format, a different serving path
than what's actually being measured below):

Both are in line with the official (non-uncensored, non-Swift)
checkpoint's own numbers on the same hardware and serving stack --
this quantization doesn't cost throughput relative to the baseline
recipe it reproduces.

Why this exists

Two other third-party attempts to combine Swift + uncensoring into an
AutoRound/GPTQ checkpoint for vLLM both fail to boot on a 24GB card
(torch.OutOfMemoryError at KV cache init, regardless of context tier).
In both cases the root cause was the same: lm_head and embed_tokens
were left unquantized at full BF16 (untied embeddings on this
248,320-vocab / 5,120-hidden model means that's ~5GB right there), and
the vision tower was similarly left unquantized. This checkpoint runs
prepare/'s int8 shrink pass on lm_head, embed_tokens, and the MTP
module on top of the AutoRound pass -- the step both of those checkpoints
skipped -- which is what makes it actually fit.

Quantization recipe

• Tool: auto-round 0.15.0
• Scheme: W4A16, group_size=128, symmetric
• Calibration: 128 samples, 200 iterations (SignRound per-block
  early-stopping -- most blocks converged well before iter 200; observed
  range iter 67-197 in this run)
• Excluded from quantization (kept BF16, matching dbirks' original
  recipe): linear_attn.in_proj_a, linear_attn.in_proj_b (DeltaNet
  recurrence-control projections), visual.* (vision tower), mtp.*
  (draft head)
• Quantized on top, via this repo's own prepare/ scripts (not part
  of the AutoRound pass): lm_head and embed_tokens to int8 g128
  (round-trip relative error 0.0064 / 0.0056), the MTP module's linear
  layers to int8 g128 (round-trip relative error 0.0066-0.0153 across its
  8 linears), plus a 40,960-token draft vocabulary built for speculative
  decoding
• Quantization tuning time: 1h48m on a single RTX 3090 (64/64
  transformer blocks, 400/607 tensors actually quantized -- the rest are
  the excluded BF16 layers above)
• On-disk size: 16.8GB (down from 52GB BF16)

Reproduction script (quantize_abliterated.py) and this exact provenance
writeup are included in this repository.

Correctness validation

Ran a 6-task battery (math, code generation, factual recall, a
constraint-logic puzzle, a security-training explanation, and strict
output-format compliance) against this checkpoint and against the
official Qwen3.8-27B-W4A16-AutoRound baseline, both served via vLLM
0.28.0 with DFlash2 MTP speculative decoding on an RTX 3090:

6/6 correct on both -- no quality regression observed from this
quantization relative to the official recipe applied to the official
checkpoint.

Reasoning-token efficiency

UkisAI's own card for the Swift adapter reports "58.3% fewer thinking
tokens" as a headline figure, with a more specific "mean thinking token
reduction of 41.0% on general reasoning benchmarks." On our 6-task
battery above, this checkpoint used 1,058 total reasoning tokens
versus 1,539 for the official baseline on the same prompts --
31.2% fewer, directionally consistent with UkisAI's own numbers but
measured independently, on a small sample, after quantization. The
efficiency gain was not uniform across tasks -- on one logic puzzle this
checkpoint used more reasoning tokens than the baseline (267 vs. 209)
-- so treat this as a real, net efficiency gain, not a strict per-task
guarantee.

The "no refusal" result on the security-training prompt is a weak
signal for the uncensoring specifically: it's a fairly mainstream
security-awareness topic that the aligned baseline also answered without
refusing. This checkpoint's refusal-rate behavior under harder
adversarial prompts has not been separately stress-tested here; see
d0xin's own INTELLIGENCE_VALIDATION.json and 100-prompt refusal eval
for that base model's own numbers (0/100 refusals reported there).

Known limitations

• Validated with a 6-prompt battery, not a full benchmark suite (no
  GPQA/MMLU-Pro/LiveCodeBench/AIME reproduction here).
• Ships with the same shared DFlash2 drafter and draft_vocab_ids.json
  used by the official-weights checkpoints -- that drafter is trained on
  the official model's output distribution, so expect a lower MTP
  acceptance rate on content this tune generates differently (~20-21%
  in this family's other uncensored variants, rejection sampling always
  falls back to this model's own distribution so it's a speed
  characteristic, not a correctness issue).
• CTX=huge (KVarN) tier tested and boots clean; both tiers were only
  exercised with short completions during validation, not
  long-context/long-generation stress testing.

Files

Model weights and tokenizer/config files are unmodified in structure
from the AutoRound + prepare/ pipeline output -- see config.json's
quantization_config for the exact compressed-tensors quantization
metadata. chat_template.jinja is byte-identical to the official
Qwen3.8-27B checkpoint's (sha256 c3cf9e34...81041).

License

Swift Open License v1.0 (LICENSE) governs the Swift Contribution this
work incorporates; commercial use above $1M annual revenue requires a
separate license from UkisAI (contact: https://ukisai.com/contact). The
underlying Qwen3.8-27B base remains under Apache License 2.0
(LICENSE-APACHE-2.0), included per Section 4(e) of the Swift Open
License v1.0. See NOTICE for the full chain.