library_name: transformers
pipeline_tag: image-text-to-text
inference: false
license: apache-2.0
base_model:
- d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
tags: - qwen3.8
- nvfp4
- w4a4
- compressed-tensors
- vllm
- sglang
- blackwell
- rtx-5090
- abliterated
- uncensored
Swift (1.0) abliterated Qwen3.8-27B NVFP4 (HF format / vLLM-ready)
HF-format quantized weights (compressed-tensors NVFP4, W4A4 group-16) of
d0xin's Swift-Qwen3.8-27B-Uncensored-BF16
(huihui-style abliteration) — this is the exact pre-conversion source of our
NInfer v3 container
Qwen3.8-27B-swift-abliterated-nvfp4full-dflash2-NInfer-v3.
Published for structured-output users: NInfer does not support JSON-schema
output; vLLM does. DFlash2 speculative decoding is engine-specific and lives
only in the NInfer container.
Sibling: dragooy's Swift-Qwen3.8-27B-abliterated-NVFP4
(a different quantization of the same base — credit to Dragoy for the
HF-format-first idea).
What's inside
model.safetensors— 17.1 GiB NVFP4 (W4A4 gs16) text stack + vision tower, W8G32 token embedding + output head; calibrated on 512 Ultrachat samples (seq 2048)model_mtp.safetensors— MTP head (used by the NInfer container; ignored by vLLM)recipe.yaml— the llm-compressor recipe, for reproducibility- Packaging note: per-packing-group global scales are unified (shrink-only E4M3 re-encode) — mathematically equivalent to the raw llm-compressor output within E4M3 re-encode precision.
Run with vLLM
pip install -U vllm
vllm serve kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF \
--max-model-len 131072 # 32 GB GPU: 131k fits comfortably; 262k on 48 GB+
JSON-schema structured output
from vllm import LLM, SamplingParams
from vllm.sampling_params import GuidedDecodingParams
llm = LLM(model="kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF", max_model_len=131072)
schema = {
"type": "object",
"properties": {"answer": {"type": "string"}, "confidence": {"type": "number"}},
"required": ["answer"],
}
sp = SamplingParams(temperature=0.0, guided_decoding=GuidedDecodingParams(json=schema))
out = llm.chat([{"role": "user", "content": "What is 2+2? Answer in JSON."}], sp)
print(out[0].outputs[0].text)
Credits & license
- All model credit to d0xin (uncensored BF16), huihui-ai (abliteration
lineage), ukisai (Swift post-training), Qwen team (base, Apache-2.0). - License: Apache-2.0, inherited through the artifact chain.
- Quantization/packaging by kaushikvira.
Benchmarks — not measured yet on this packaging
The quantization is identical to our NInfer v3 container of the
same weights (which carries full gate/needle/llama-benchy numbers on a single RTX 5090), but the
vLLM/SGLang-specific numbers for this repo are not measured yet — this is a fresh packaging.
If you run it, please share your numbers in the Community tab (tool + version, GPU, context
length, pp/tg tok/s) — we'll collect them here for everyone.