license: other
language:

• en
  base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-FP8
  tags:
• qwen
• qwen3.5
• qwen3.8
• text-generation
• multimodal
• vllm
• speculative-decoding
• compressed-tensors
• w4a16
• autoround
• uncensored
• nsfw
• image-text-to-text
  pipeline_tag: image-text-to-text

Swift-Qwen3.8-27B-Uncensored-W4A16 (AutoRound)

The uncensored "Swift" Qwen3.8-27B checkpoint, quantized from FP8 to a
4-bit W4A16 compressed-tensors format with AutoRound, ready to serve
on vLLM with DFlash2 speculative decoding.

This is the model that powers the qwen3.8-swift-quetza-vllm local launch
profile. It runs as-is on a 24 GB card (RTX 3090/4090-class) at a
200,000-token context using int4 KV cache.

Gated repo. Download requires accepting the license / requesting access
on the repo page (the source FP8 model is also gated).

What this is

Build recipe

Mirrors the W4A16 AutoRound card this was based on:

• scheme: W4A16 (weights int4, activations fp16)
• dataset: NeelNanda/pile-10k
• nsamples 128, seqlen 2048, batch_size 4, iters 200, seed 42
• format: llm_compressor → compressed-tensors

⚠ Size note (read this before downloading)

This checkpoint is larger than the 4-bit base profile even though both are
W4A16 g128. AutoRound left embed_tokens and lm_head in BF16
(~2.54 GB each) because its supported_types = (nn.Linear, Conv1D) does not
include nn.Embedding, so a --layer_config for those two is silently
ignored. Result:

With the DFlash2 drafter loaded, the model OOMs by ~120 MiB at full
config on a 24 GB card. Mitigations (handled by the launcher script):

• MTP tensors un-indexed (model_extra_tensors.safetensors moved aside)
• DFLASH_TOKENS lowered to 3 (from 7)
• --no-draft runs the model alone at reduced context — always fits, no
  speculative decoding

If a future Swift checkpoint has embed_tokens/lm_head 4-bit packed, these
caveats no longer apply.

Running it

The companion launcher (not in this repo) is
qwen3.8-swift-quetza-vllm, a docker/vLLM wrapper:

qwen3.8-swift-quetza-vllm                       # start vLLM + QuetzaCodetl
qwen3.8-swift-quetza-vllm --no-draft            # if the load OOMs (frees ~3.5 GB)
qwen3.8-swift-quetza-vllm --no-reasoning        # thinking OFF
qwen3.8-swift-quetza-vllm --status             # backend state
qwen3.8-swift-quetza-vllm --stop               # free the GPU

It uses the image ghcr.io/syv-ai/qwen38-27b-rtx3090:latest, serves on
localhost:18022 as model name qwen3.8-27b, int4 KV cache, 200K context.

Files

• model-0000X-of-00007.safetensors — quantized weights (7 shards)
• model_extra_tensors.safetensors — MTP / extra tensors (un-indexed by the launcher)
• model.safetensors.index.json — weight map
• config.json, quantization_config.json, generation_config.json
• tokenizer.json, tokenizer_config.json, chat_template.jinja
• preprocessor_config.json, processor_config.json — vision processor