license: other
license_name: swift-open-license-1.0
license_link: LICENSE
base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• qwen3.8
• swift
• abliterated
• uncensored
• w4a16
• 4-bit
• int4
• int8
• gptq
• compressed-tensors
• autoround
• vllm
• marlin
• mtp
• speculative-decoding
• efficient-thinking
• reasoning
• function-calling
• vision-language
• rtx-3090

Swift-Qwen3.8-27B-Uncensored-W4A16-fast

The "fast variant" of
Swift-Qwen3.8-27B-Uncensored-W4A16:
the same 4-bit body of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
(UkisAI's reasoning-efficient Swift-Qwen3.8-27B
with its refusal direction removed), with the lm_head and the MTP module requantized to
int4 GPTQ. It serves on one 24 GB GPU (RTX 3090 class) with the
HyperQwen stack. The int4 lm_head makes every
decode step read about 0.65 GB less; that is where the gain over the base build comes from.

Lineage

Qwen/Qwen3.8-27B                              Apache 2.0
 └─ ukisai/Swift-Qwen3.8-27b                  Swift Open License v1.0, reasoning-efficiency LoRA merged
     └─ d0xin/Swift-Qwen3.8-27B-Uncensored-BF16   rank-1 directional ablation, layer 38
         └─ ...-W4A16                         W4A16 g128 body, int8 lm_head / embed_tokens / MTP
             └─ this model                    int4 GPTQ lm_head / MTP

What changed against the base build

Relative error is ||dequant - w|| / ||w|| against the BF16 source weights. GPTQ uses
300,000 hidden states captured from this model's own generations, so it minimizes the
output error on real activations rather than the weight error that this number shows.
On those activations the KL divergence to the BF16 lm_head is 0.00251 for GPTQ
int4, against 0.00558 for round-to-nearest int4. All groups are symmetric (no zero
points).

The MTP module was evaluated on a held-out split of the captured data (greedy chain
simulation, 2 draft positions): 2.269 tokens per step, first-position top-1
agreement 0.777, acceptance 0.845, n = 107,063 positions.

The draft vocabulary is the top 40,960 ids of 6,761 generations by this model (4.59 M
output tokens, 3,119 of them with thinking on; prompts: English chat, code, Danish
instructions and reasoning, GSM8K train). On the held-out 10 % of those generations it
covers 97.68 % of tokens; the generic list that the official
Qwen3.8-27B-W4A16-AutoRound checkpoint uses covers 98.18 %. The draft head only
matters for SPEC=mtp.

How it was made

1. drafter/collect_prompts.py, drafter/gen_data.py, drafter/capture.py: generate
   with this model and capture its final hidden states.
2. drafter/gptq_lm_head.py --bits 4 --calib-rows 300000: GPTQ int4 lm_head.
3. drafter/train_mtp.py --eval-only 1 --dump-hessians, then
   drafter/requant_mtp_gptq.py --bits 4: GPTQ int4 MTP module.
4. prepare/build_draft_vocab.py: the draft head from the own-output id list.

The tools are in HyperQwen (drafter/, prepare/).

Properties of the BF16 source

These come from the source and were not measured again on this quantization:

• Refusals (d0xin, fixed 100-prompt set): 88 direct answers, 10 answers with a safety
  note, 2 other failures, 0 refusals.
• Capability (d0xin, fixed 298-example set against the original Swift BF16):
  39.93 % against 38.26 % combined, +1.68 points, McNemar p = 0.44, no measurable loss.
• Method: rank-1 directional residual-stream ablation at layer 38; 131 of 1,199
  tensors changed, vision tensors unchanged. Details in the
  source repository.
• Shorter reasoning (UkisAI): our GSM8K run shows 355 answer tokens on average
  against 379 for the official AutoRound base, with thinking off.

Measured results (single RTX 3090, 24 GB)

All numbers use the HyperQwen stack on vLLM 0.28.0, WSL2 + Docker, thinking off
(protocol v2).

Single-user decode (bench/run_benchmarks.sh single), 2026-09-23, card at 250 W,
SPEC=dflash2 (DFlash2 drafter, 7 draft tokens), PREFIX_CACHE=1, KV_MEM=4529848320
(4.22 GiB), MAX_LEN=49152. End-to-end throughput in tok/s, sampling T=default / T=0.
One run per checkpoint after a warm-up request.

C1 accepts 3.67 / 3.75 tokens per verify step, mean time to first token 184 ms. Expect
3–5 % variation between sessions.

Batch serving (bench/run_benchmarks.sh batch --prefill --long), 2026-09-21, card at
350 W, KV=fp8, no speculation, second of two runs.

Quality (bench/quality_battery.py against the served model): perplexity over
~300-token windows of wikitext-2 (en), fineweb-2 Danish (da) and Python source (code);
GSM8K exact match, first 200 test questions, greedy, thinking off.

The int4 heads cost 0.5 % perplexity. With n=200 the GSM8K standard error is about
1.2 points at this accuracy, so the one-point GSM8K difference is within noise.

How to serve

HyperQwen (Linux or WSL2, one 24 GB GPU), in .env:

MODEL=/app/models/Swift-Qwen3.8-27B-Uncensored-W4A16-fast
SPEC=dflash2          # or SPEC=mtp to draft with the int4 MTP head
PREFIX_CACHE=1

then docker compose --profile single up -d. bash verify.sh --no-server checks the
directory before you serve it.

Plain vLLM (0.28 or later) loads the body and heads through compressed-tensors:

vllm serve TyroneNel/Swift-Qwen3.8-27B-Uncensored-W4A16-fast \
  --max-model-len 32768 --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

The truncated MTP draft head (mtp.draft_lm_head.* with mtp_draft_vocab_ids.pt) is
read by HyperQwen's patches/qwen3_5-mtp-draft-vocab.patch. It is not tested on
unpatched vLLM.

Sampling defaults (generation_config.json, from Swift): temperature 1.0, top_p 0.95,
top_k 20, min_p 0, repetition_penalty 1.0. For thinking off, Qwen recommends
temperature 0.7, top_p 0.8, presence_penalty 1.5. The chat template accepts
reasoning_effort low, medium and xhigh (default), and maps minimal to low
and high / max to xhigh.

Files

Safety

This is a refusal-reduced model. It can produce content that the upstream model refuses
or handles with more care, and its output can be wrong, offensive or unsafe. Use it for
research, development and other lawful purposes. Review its output where an error can
cause harm. You are responsible for how you use it and for compliance with the licenses
and the law. Fine-tuning should start from the BF16 source, not from these 4-bit weights.

License

The weights are distributed under the Swift Open License v1.0 (LICENSE). It
permits use, modification and redistribution. Commercial use is licensed only while
you, together with every entity that controls, is controlled by or is under common
control with you, had gross revenue below US$1,000,000 in the most recently completed
fiscal year (Section 5). Above that threshold, commercial use needs a Swift Enterprise
License from UkisAI. The base model's Apache License 2.0
is included as LICENSE-APACHE-2.0 (Section 4(e)); NOTICE lists the attribution
chain, UkisAI's notices and every change. This summary is not legal advice; LICENSE
is the binding text.

Original Swift model: UkisAI. Abliteration: d0xin. Quantization and serving preparation:
TyroneNel.