license: other
license_name: swift-open-license-1.0
license_link: https://ukisai.com/contact
base_model: ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP
base_model_relation: quantized
library_name: transformers
language:

• en
• zh
  pipeline_tag: image-text-to-text
  tags:
• gptq
• int4
• uncensored
• abliterated
• mtp
• speculative-decoding
• intel
• arc
• intel-arc
• xpu
• vllm
• qwen3.8
• swift

Swift-1.5-Qwen3.8-27B-Uncensored-GPTQ-Int4-sym-G128-MTP-BF16

GPTQ-Int4 quantization of ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP, built for
Intel Arc / vLLM XPU with the MTP head preserved in BF16 so native
speculative decoding still works.

The recipe is a reproduction of kernelogic/Qwen3.8-27B-Uncensored-GPTQ-Int4-sym-G128-MTP-BF16,
applied to the Swift lineage instead of the JonathanColetti one. Same quantizer
version, same config, same MTP handling.

Why this exists

Quantizing this model naively breaks speculative decoding. The 15
mtp.* tensors are the draft head; if they get quantized along with everything
else, draft acceptance collapses and you lose roughly half your decode speed.
The fix is one line of quantize config:

dynamic={"-:.*mtp.*": {}}   # exclude mtp.* from quantization -> stays BF16

The result has 400 quantized weight tensors (I32) +
15 preserved BF16 MTP tensors.

There is a second, less obvious requirement. transformers' Qwen3_5 declares
_keys_to_ignore_on_load_unexpected = [r"^mtp.*"], so the draft tensors are
dropped at load and never become modules — the dynamic exclusion on its own
emits a checkpoint with no draft head at all, silently. What writes them back is
gptqmodel's out_of_model_tensors = {"prefixes": ["mtp"]}, which re-reads them
from the source safetensors at save time. That is also why they stay BF16 while
the body is fp16: they are copied bytes, never converted.

Provenance

Qwen/Qwen3.8-27B                          (base)
  └─ ukisai/Swift-1.5-Qwen3.8-27b                 (Swift 1.5, reasoning-efficient finetune)
       └─ ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP
            (uncensored, orcarouter rank-1 refusal direction projected out, BF16)
              └─ this repo                             (GPTQ-Int4, 20 GB)

Quantization

Group size is 128, verified from the tensor shapes
(qweight.rows * 8 / scales.rows) rather than taken from the config's own
declaration. lm_head and the vision tower (333 tensors) are left
unquantized.

Measured performance

Not yet measured on this build. The structural properties above are read
from the checkpoint; throughput and MTP acceptance are not derivable from one
and have to be measured on the target hardware.

For reference, the checkpoint this recipe reproduces reports 68.5
tok/s on code generation and 52.4 on prose (single Arc Pro B70, 230 W, MTP4,
fp8 KV, 131,072 context) — that is a different checkpoint on a different
serving stack, so treat it as an order of magnitude, not a prediction for this
one.

Throughput on this architecture tracks MTP acceptance, which depends on how
predictable the output is: code drafts well, freeform prose less so. Any number
quoted without an acceptance figure alongside it is not reproducible.

Serving (vLLM XPU)

vllm serve /model \
  --quantization gptq --dtype float16 \
  --kv-cache-dtype fp8 \
  --max-num-batched-tokens 8192 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --chat-template /model/chat_template.jinja

Qwen3.8 MTP on XPU needs a patched vLLM. The build used to produce and serve
this checkpoint is at
github.com/greglechin/vllm-xpu; the
Intel Arc Pro B70 cookbook
is the other published route. Context length, --tensor-parallel-size and
--gpu-memory-utilization are deliberately omitted above because they depend on
your card count and VRAM — size them from your own boot log.

Three things that will bite you

1. MTP depth is not free, and tok/s is the wrong way to judge it. Read the
   per-position acceptance curve from the engine's SpecDecoding metrics, not
   the run mean: a decay ratio around 0.8 pays for the extra drafter pass and
   around 0.68 is break-even. Changing depth typically moves throughput by less
   than boot-to-boot variance, so a single A/B run measures nothing.
2. Prefix caching depends on your engine build. Qwen3.8 is a hybrid
   GDN/Mamba architecture and upstream vLLM does not declare
   supports_mamba_prefix_caching for it, which is why the reference checkpoint
   advises disabling it (measured there: 3,036 queries → 0 hits). With the
   mamba-align patches and --mamba-cache-mode align it does work — 84.8–85.3%
   hit rate measured on the serving stack above. Check your own hit-rate metric
   rather than assuming either outcome.
3. --kv-cache-dtype fp8 is effectively required on 32 GB cards at long
   context; fp16 KV does not fit.

Limitations — please read

• No quality evaluations were run on this quantization. No perplexity
  comparison against the BF16 source, no coding or reasoning benchmarks, no
  quantitative refusal-rate testing. What was verified is structural: tensor
  counts and dtypes, the preserved BF16 MTP head, complete quantized-module
  tensor sets, group size derived from shapes, and an unquantized lm_head and
  vision tower. If you need quality guarantees, measure before relying on it.
• Calibration used general web text with no code. If coding quality matters
  to you, a code-inclusive calibration set would likely be better. This is
  inherited from the reference recipe, which names the same limitation.
• Uncensoring is inherited, not verified here. All refusal-removal
  properties come from the upstream ablation (ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP
  publishes its own refusal evaluation); this repo only changes numeric
  precision. Quantization was not evaluated for its effect on that behaviour in
  either direction.
• The upstream publishes a KL divergence and states what it did not measure. It reports KL 0.0884 against ukisai/Swift-1.5-Qwen3.8-27b on first-token distributions over 100 harmless prompts, and 23/100 refusals, both under Heretic's built-in evaluation with the reference rows measured the same way. Its refusal direction is recovered from orcarouter's published weights rather than fitted fresh, and it verifies that recovery by reproducing orcarouter's 131 edited tensors to 99.75% bit-identical. What it explicitly does not evaluate: general benchmarks, refusal behaviour in thinking mode, whether Swift 1.5's shorter reasoning traces survive, and MTP acceptance — which on a speculative-decoding deployment is the number that decides decode throughput. Measure acceptance on your own traffic before promoting this over a checkpoint you have already gated.
• This is an uncensored model. It will attempt requests an aligned model
  declines. You are responsible for how you use it.

Licence

Not Apache-2.0. These weights inherit the Swift Open License v1.0 from
ukisai/Swift-1.5-Qwen3.8-27b. Personal, research, educational, evaluation
and commercial use are free for individuals and organizations with annual
recurring revenue, including affiliates, of up to US$1,000,000. Above that
threshold, commercial use requires a separate Swift Enterprise License — contact
UkisAI.

Credits

• Qwen — base model
• UkisAI — Swift finetune
• ajgazin — directional ablation
• kernelogic/Qwen3.8-27B-Uncensored-GPTQ-Int4-sym-G128-MTP-BF16 — the quantization recipe this reproduces
• gptqmodel — quantization