license: other
license_name: swift-open-license-1.0
license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b/blob/main/LICENSE
library_name: ninfer
pipeline_tag: image-text-to-text
inference: false
base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
base_model_relation: quantized
tags:
- qwen3.8
- swift
- uncensored
- nvfp4
- fp8
- mixed-precision
- local-hessian
- activation-headroom
- ninfer
- dflash2
- vision
- multimodal
- blackwell
- sm_120a
- not-for-all-audiences
Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer
Native NInfer build of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16, re-quantized from BF16 using NVIDIA ModelOpt Local-Hessian calibration + activation headroom and a conservative mixed NVFP4/FP8 topology.
Release goals
The goal of this release is straightforward: retain Swift-Qwen3.8's
capabilities while removing refusal behavior.
It is designed to preserve Swift-Qwen3.8's reasoning, agentic, tool-calling,
multimodal and long-context capabilities while removing refusal behavior.
For this quantized build specifically, the goal is also to increase inference
throughput and reduce VRAM usage while preserving FP8-level measured quality
and avoiding longer reasoning traces under the recommended B2048 serving
profile.
In our fixed validation, this build delivered +20–27% decode throughput,
+34–38% prefill throughput, and 15.1% lower VRAM usage versus FP8, while
scoring 225/280 vs 224/280 on the B2048 MMLU-Pro subset. Mean reasoning
usage was also lower: 741.9 vs 793.0 tokens.
What makes this build different
This is not a repack of an existing NVFP4 checkpoint.
| Component | Precision |
|---|---|
| MLP layers 0-55 | NVFP4 W4A4 |
| MLP layers 56-63 | row-wise FP8 |
| Self-attention projections | row-wise FP8 |
| Linear-attention / GDN projections | row-wise FP8 |
| GDN a/b | BF16 |
| LM head | row-wise FP8 |
| Embedding | row-wise FP8 |
Calibration used NVIDIA ModelOpt Local-Hessian, activation headroom, FP8 scale sweep, layer-wise checkpointing, upper_percentile=99.99, and rho=16384.
Quality validation
Fixed MMLU-Pro subset: 280 questions, 14 categories, 20 questions per category, seed 20260918, temperature=0, reasoning_effort=xhigh, max_tokens=4096.
| Build | Correct | Accuracy |
|---|---|---|
| FP8 | 211/280 | 75.36% |
| NVFP4 Headroom | 207/280 | 73.93% |
| FP8 + B2048 | 224/280 | 80.00% |
| NVFP4 Headroom + B2048 | 225/280 | 80.36% |
For FP8+B2048 vs this build+B2048: both correct 216, both wrong 47, FP8-only correct 8, NVFP4-only correct 9, exact McNemar p=1.0.
No statistically significant accuracy difference was observed on this subset.
Recommended thinking budget
Recommended NInfer setting: --default-thinking-budget 2048.
Without a thinking budget, 36/280 NVFP4 evaluation generations reached the 4096-token output ceiling. With B2048, reasoning median was 379.5 tokens, p90 was 2072, mean was 741.87, 0/280 reached the 4096-token limit, all 280 requests finished normally, and accuracy was 225/280.
The thinking budget is a serving recommendation and is not baked into the artifact.
Performance vs FP8
Test hardware: NVIDIA RTX PRO 6000 Blackwell Workstation Edition.
| Metric | FP8 | This build | Change |
|---|---|---|---|
| Decode C1 | 134.87 tok/s | 162.32 tok/s | +20.4% |
| Decode C8 | 87.33 tok/s | 110.58 tok/s | +26.6% |
| Prefill C1 | 6633.88 tok/s | 8876.31 tok/s | +33.8% |
| Prefill C8 | 6960.00 tok/s | 9615.82 tok/s | +38.2% |
| NInfer VRAM | 41358 MiB | 35106 MiB | -15.1% |
Performance is hardware- and workload-dependent.
Refusal evaluation
Fixed Swift ORCA-K1 100-prompt evaluation:
| Classification | Count |
|---|---|
| DIRECT | 90 |
| SAFETY_DEFLECT | 10 |
| REFUSE | 0 |
| OTHER_FAILURE | 0 |
Configuration: temperature=0, seed=42, max_tokens=1536, thinking disabled.
Historical FP8 responses re-judged with the same current judge produced 87 DIRECT, 12 SAFETY_DEFLECT, 1 REFUSE, and 0 OTHER_FAILURE.
The 0/100 refusals result applies only to this fixed automated evaluation. It is not a universal non-refusal guarantee.
Safety and responsible use
This model has deliberately reduced refusal behavior.
Reduced refusal behavior also reduces model-level safety filtering.
Users are responsible for evaluating the model for their own deployment,
safety, legal, and compliance requirements.
The 0/100 figure describes the documented fixed evaluation set and
inference configuration. It should not be interpreted as a guarantee that
no conceivable prompt, sampling configuration, system prompt, or inference
engine can ever produce a refusal.
Artifact
- File:
Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer - Size:
23,719,715,844 bytes - SHA-256:
a298e28529a31f5447574ba2ad5ba5812d952053292ca017115f9d735dfdadcd - Stored objects:
1,246
The artifact contains Text, Vision, MTP, DFlash2, the optimized proposal head, and tokenizer/frontend resources.
Runtime requirements
Validated NInfer revision: f76e19c0fbd026c86f46005acf2c80c54084bade.
Current NInfer targets NVIDIA Blackwell sm_120a. This build was validated on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition.
The current .ninfer runtime should not be considered portable to Ampere, Ada, or Hopper. This limitation primarily comes from the current NInfer execution target, not from the conceptual NVFP4 quantization format itself.
Recommended NInfer server
ninfer-serve \
./Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer \
--host 0.0.0.0 \
--port 8080 \
--model-id Swift-Qwen3.8-27B-Uncensored-NVFP4 \
--max-context 262144 \
--kv-capacity 262144 \
--max-concurrency 8 \
--kv-dtype fp8 \
--prefill-chunk 1024 \
--spec dflash2 \
--draft-tokens 5 \
--lm-head-draft \
--preserve-thinking \
--default-thinking-budget 2048 \
--vision
--max-context 262144 is the logical ceiling for one request.
--kv-capacity 262144 is a shared Main Text KV pool across concurrently admitted requests. It does not allocate 262144 KV tokens independently to each concurrent request.
Toolchain
- NInfer:
f76e19c0fbd026c86f46005acf2c80c54084bade - ModelOpt:
9e3d555aa16038f7b46a631ca603351146c09123 - llmcompressor:
6693441a7db6984415d524f51ed16a1605bf2fec - compressed-tensors:
0.18.1.a20260914
Model lineage
Qwen/Qwen3.8-27B
↓
ukisai/Swift-Qwen3.8-27b
↓
d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
↓
Local-Hessian + Activation Headroom
mixed NVFP4/FP8
↓
this NInfer artifact
Limitations
Quantization can change individual generation trajectories even when aggregate benchmark accuracy is similar.
The MMLU-Pro result reported here uses a fixed 280-question subset, not the full benchmark. The refusal evaluation uses a fixed automated 100-prompt set. Performance results were measured on one RTX PRO 6000 Blackwell system and should not be interpreted as universal performance values.
License
This model is a derivative of ukisai/Swift-Qwen3.8-27b.
Swift-specific contributions are distributed under the Swift Open License v1.0. The underlying Qwen base model is distributed under Apache License 2.0.
See the included LICENSE, LICENSE-APACHE-2.0, and NOTICE files for applicable terms and attribution requirements.
Attribution
- Base model:
Qwen/Qwen3.8-27B - Swift fine-tune:
ukisai/Swift-Qwen3.8-27b - Uncensored derivative, quantization, NInfer conversion, and validation:
d0xin