license: other
license_name: swift-open-license-1.0
license_link: https://huggingface.co/Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4/blob/main/LICENSE
base_model: ukisai/Swift-Qwen3.8-27b
tags:

• nvfp4
• fp8
• qwen3.8
• qwen3_5
• abliterated
• uncensored
• multimodal
• mtp
• vllm
• transformers
• blackwell
  pipeline_tag: image-text-to-text
  library_name: transformers

Swift-Qwen3.8-27B · huihui-style abliterated · NVFP4 (standard HF format)

NVFP4 + FP8 quantized checkpoint of the abliterated Swift model, published in the
standard Hugging Face safetensors format. It loads in vLLM and
transformers, so standard features such as JSON-schema structured output
(vLLM guided decoding) work out of the box.

Specs

The NVFP4 weights alone are 21.8 GiB, so a single 16 GB card is not enough for
this format. Working options for 16 GB-class cards:

1. GGUF IQ4_XS (~14.5 GiB, imatrix, MTP head included) —
   jakeatx/ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M-GGUF
   (abliterated Swift, text-only) via llama.cpp / LM Studio.
   For 24 GB cards with long context the same author documents a dedicated recipe.
2. 2 × 16 GB, tensor parallel 2 — NVFP4 runs on 2 × RTX 5070 Ti
   (16 GB each): documented benchmark at
   unsloth/Qwen3.8-27B-NVFP4, discussion #16
   (54–120 tok/s decode with MTP, 80–120K context).
3. NVFP4 + CPU offload — vLLM --cpu-offload-gb / llama.cpp partial -ngl:
   full 1:1 quality, lower speed (15–40 tok/s range).
4. The hybrid mamba/attention architecture keeps the KV cache minimal, so the
   weight size — not the context — is the binding constraint on small cards.

Usage (vLLM)

pip install vllm==0.27.1
export VLLM_USE_FLASHINFER_SAMPLER=0   # no nvcc needed in the container
vllm serve dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 \
  --max-model-len 32768 --gpu-memory-utilization 0.9

JSON-schema output:

from vllm import SamplingParams
params = SamplingParams(temperature=1.0, top_p=0.95, top_k=20,
                        guided_json={"type": "object",
                                     "properties": {"answer": {"type": "string"}},
                                     "required": ["answer"]})

Provenance

1. Source: ukisai/Swift-Qwen3.8-27b @ 1b30aaaf753fe5c1cb51ada2ea0367a53445359c
2. Abliteration: huihui-style (refusal-direction projection, Arditi et al. 2024 family),
   transferred by per-tensor weight difference from the Qwen/Qwen3.8-27B ↔
   huihui-ai/Huihui-Qwen3.8-27B-abliterated pair — format/weight change, no fine-tuning
3. Quantization: llm-compressor oneshot, NVFP4 + FP8 allocation copied verbatim from
   unsloth/Qwen3.8-27B-NVFP4 (quantization_config), 32 calibration samples
4. Config repair: nested multimodal config restored + quantization_config grafted
   (llm-compressor flattens it)
5. Verification: shape preflight + vLLM 0.27.1 smoke on RTX PRO 6000

Also available

The same checkpoint, packaged as a single-file engine artifact for Blackwell
(sm_120a) instead of standard safetensors:
Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-NInfer.

License

This repository is a derivative of the Swift checkpoint, whose license is the
Swift Open License v1.0 — not Apache. The chain:

What that means in practice:

• Free use, including commercial, while your gross revenue (counting all
  controlled entities) is below the $1,000,000 per fiscal year threshold;
  qualified non-profits have no threshold for non-commercial or research use.
• Above the threshold: obtain a separate written licence from UkisAI
  (Swift Enterprise License).
• Redistribution: ship both licence files, keep the copyright and attribution
  notices, and mark files you modified (Swift licence §4–§5).

This is a description of what the licences say, not legal advice.

Intended use and limitations

This is an uncensored model: the upstream abliteration removes the refusal
direction, so it will attempt requests a stock instruction-tuned model declines.
It is published for research, evaluation and local deployment where that
behaviour is understood and wanted.

Use at your own responsibility. Anyone deploying it is responsible for their
own safeguards, output handling, and compliance with the licences above and
applicable law. The publisher publishes this checkpoint as-is, without warranty
of any kind, and is not liable for how it is used or for its outputs. The refusal
behaviour — and any capability cost of the abliteration — originates
entirely upstream; this repository changes precision and format, not behaviour.

Credit for the base model to Qwen (Alibaba Cloud);
for the Swift training to UkisAI; for the
abliteration style to huihui-ai; and for the
published NVFP4 recipe to unsloth.