license: other
license_name: swift-open-license-1.0
license_link: LICENSE
base_model: greglechin/Swift-1.5-Qwen3.8-27B-Uncensored-GPTQ-Int4-sym-G128-MTP-BF16
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags: [qwen3_8, swift, uncensored, abliterated, gptq, int4, mtp, dflash2, speculative-decoding, intel, xpu, arc]
Swift-1.5 Qwen3.8-27B Uncensored, GPTQ Int4 baked
An uncensored Swift 1.5 (Qwen3.8-27B) that one Intel Arc Pro B65 32 GB serves at the full
262,144-token context with the lm_head and the MTP draft already INT4 on disk, so stock
vLLM loads it with no weight patches. Paired with
Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter
it decodes at 96.7 tok/s single-stream on that card (BetterBench weighted median; MTP alone: 80.5).
Lineage (every step credited in NOTICE):
Qwen/Qwen3.8-27B →
ukisai/Swift-1.5-Qwen3.8-27b (UkisAI's reasoning-efficient post-training) →
ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP (refusal-direction ablation, MTP head edited consistently) →
greglechin/Swift-1.5-Qwen3.8-27B-Uncensored-GPTQ-Int4-sym-G128-MTP-BF16 (GPTQ INT4 body, head and MTP left BF16) →
this repo: head and MTP draft re-quantized to GPTQ INT4 with Hessians captured inside the serving engine.
What changed
| tensor | greglechin | this repo |
|---|---|---|
| 400 body linears | GPTQ INT4 g128 | byte-identical (sha256 in BAKE.json) |
lm_head (248,320 × 5,120) |
BF16, 2.54 GB | GPTQ INT4 sym g128 on disk, 0.66 GB |
mtp.* draft: fc, q/k/v/o, gate/up/down |
BF16, 0.85 GB | GPTQ INT4 sym g128 on disk, 0.22 GB |
MTP norms, embedding, vision tower, GDN in_proj_a/in_proj_b |
BF16 | unchanged |
17.05 GB on disk; 15.9 GiB resident in vLLM, which leaves the KV budget for 262k context on 32 GB.
How the bake was done. The dense checkpoint was served in vLLM 0.30.0 with MTP on, and forward
hooks accumulated the input second-moment matrices of the nine linears over three generated slices
(256 wikitext prompts, 256 UltraChat conversations through the chat template, 128 CodeParrot prompts,
128 greedy tokens each; ~290k rows for lm_head), every 64th row held out. Each linear was GPTQ-quantized
for each of three dampings × three slice mixes and the setting with the lowest relative output error on
the held-out rows was packed in the body's GPTQ layout. Relative output L2 error on the same held-out
rows, vs the equivalent tensors of Wrapzii's Swift 1.5 bake and vs round-to-nearest:
| linear | this bake | Wrapzii bake | RTN |
|---|---|---|---|
lm_head |
4.81% | 6.34% | 7.97% |
mtp.fc |
7.18% | 9.73% | 11.39% |
mtp.layers.0.self_attn.q_proj |
2.70% | 3.51% | 4.39% |
mtp.layers.0.self_attn.k_proj |
6.09% | 7.95% | 9.99% |
mtp.layers.0.self_attn.v_proj |
5.12% | 6.71% | 8.16% |
mtp.layers.0.self_attn.o_proj |
6.68% | 9.02% | 13.07% |
mtp.layers.0.mlp.gate_proj |
3.76% | 4.91% | 6.36% |
mtp.layers.0.mlp.up_proj |
5.84% | 7.62% | 9.58% |
mtp.layers.0.mlp.down_proj |
7.81% | 11.31% | 12.12% |
(o_proj and down_proj of the MTP draft are not like-for-like with Wrapzii's: ajgazin's ablation edits those two tensors.)
Measurements (Arc Pro B65 32 GB, vLLM 0.30.0 XPU, fp8 KV, single stream)
BetterBench 0.4.0, quick run (1 warmup + 5 passes per category), category-weighted median decode tok/s:
| configuration | weighted decode | tokens / step | prefill @2K |
|---|---|---|---|
| MTP k=6 (built-in draft), 262k ctx | 80.5 | 3.35 | 1294 |
| DFlash2 INT4 drafter k=7, 128k ctx | 96.7 | 4.23 | 1290 |
| reference: Wrapzii/Swift-1.5-…-baked-v1-embed-int8 (not abliterated), MTP k=6 | 88.6 | 3.68 | 1274 |
With the drafter attached the KV budget on 32 GB tops out below 262k (a 262k request needs 11.5 GiB of KV, 7.9 GiB are
free), so the DFlash2 rows are at 128k context (157,860-token KV pool); MTP mode serves the full 262k. DFlash2 wins
single-stream and at concurrency 4 (184 vs 169 aggregate tok/s) but not at concurrency 16 (197 vs 232), where the
extra drafter pass per step outweighs acceptance.
Quick runs are not publishable numbers; the Launch80 pages (https://launch80.com/a/a22029b0-497e-49e7-9f48-0677f1789da0 , https://launch80.com/a/602f86d7-b0d3-4d2b-9c94-1037fe1d3cd3) render the raw results,
and the JSON files are the citation of record.
Quality gates (same card, same engine):
| gate | MTP arm | DFlash2 arm | dense-head reference |
|---|---|---|---|
| perplexity, 13k-token mixed corpus | 6.200 | same target | 6.186 |
| perplexity, wikitext-2 test 40×2048 | 6.216 | same target | 6.188 |
| GSM8K n=100, 1536-token budget | 94 | 94 | |
| refusals on 50 XSTest safe prompts | 0 | 0/50 | |
| greedy spec-on == spec-off, 16 prompts | 15/16 (one 0.016-nat tie) | 16/16 |
Speculative decoding is lossless by construction; the greedy check guards the draft/verifier path.
Serving
# MTP (built-in draft), stock vLLM ≥ 0.30 on XPU or CUDA
vllm serve Launch80/Swift-1.5-Qwen3.8-27B-Uncensored-Int4-baked \
--quantization gptq --dtype float16 --kv-cache-dtype fp8 --max-model-len 262144 \
--speculative-config '{"method":"mtp","num_speculative_tokens":6}'
# DFlash2 INT4 drafter (faster): bf16 drafter is required (the drafter overflows in fp16)
vllm serve Launch80/Swift-1.5-Qwen3.8-27B-Uncensored-Int4-baked \
--quantization gptq --dtype bfloat16 --kv-cache-dtype fp8 --max-model-len 131072 \
--speculative-config '{"method":"dflash","model":"Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter","num_speculative_tokens":7,"quantization":"gptq"}'
On Intel XPU the fp16 target is a little faster than bf16; keeping the target fp16 while the drafter runs in
bf16 needs the small boot patch patch_dflash_mixed_dtype.py from the launch80/B65
repository, which also holds the three vLLM-XPU correctness patches (MTP nightly, MTP boundary, GDN mixed-split)
used for every number above. The INT8-embedding side file of the Wrapzii lineage is not used here.
License
Not Apache-2.0. UkisAI's Swift contribution is under the Swift Open License v1.0 (LICENSE): free for
personal, research and commercial use by entities under US$1,000,000 annual revenue; above that a Swift
Enterprise License from UkisAI is required. The Qwen3.8-27B base is Apache-2.0
(LICENSE-APACHE-2.0). NOTICE carries the attribution chain and MODIFICATIONS.json the change notice.
Refusal behaviour was removed upstream by ajgazin's ablation; you are responsible for your use.
Credits
Qwen (base model) · UkisAI (Swift 1.5) · ajgazin (abliteration with the MTP head kept) · greglechin (GPTQ body for Arc/XPU)
· z-lab / Inco AI (DFlash 2 drafter) · Wrapzii (the on-disk bake layout this follows, with Launch80's bake tooling) ·
davetha (the Huihui bake that started the TriplePoint comparison).