library_name: ninfer
pipeline_tag: image-text-to-text
inference: false
license: apache-2.0
base_model:

• d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
• ukisai/Swift-Qwen3.8-27b
• z-lab/Qwen3.8-27B-DFlash2
• huihui-ai/Huihui-Qwen3.8-27B-abliterated
  base_model_relation: quantized
  tags:
• ninfer
• ninfer-v3
• qwen3.8
• qwen3_5
• nvfp4
• w4a4
• blackwell
• multimodal
• conversational
• cuda
• rtx-5090
• dflash2
• speculative-decoding
• abliterated
• uncensored

Qwen3.8-27B Swift-abliterated nvfp4full + DFlash2 for NInfer — v3 container

An uncensored (abliterated), all-NVFP4 Qwen3.8-27B artifact for the
NInfer engine, built end-to-end on a single
RTX 5090: d0xin's abliterated BF16 checkpoint → our own NVFP4 quantization
pipeline → DFlash2 speculative drafter + indexed proposal head, packaged as a
native v3 .ninfer container.

It is the uncensored counterpart to
kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3
and replaced it as our production profile after a same-session A/B (below).

Credits & provenance (full chain)

This artifact exists only because of the following models and their authors — all credit to them:

The abliteration itself is d0xin's work, inherited byte-for-byte — no
re-abliteration, no behavioural editing was performed here. Per
ABLITERATION.json: OrcaRouter-compatible rank-1 residual-writer directional
ablation, W' = W − r(rᵀ W), 131 modified tensors, direction layer reference
38, rank-1, fp32 projection math, max BF16 residual leakage 1.49e-3. d0xin's own
paired evaluation (298 samples) measured MMLU-Pro +6.1 pts vs base (not
significant, CI crosses 0) and math-500 unchanged.

What we did (the technique)

1. All-NVFP4 quantization of the BF16 checkpoint with llm-compressor
   (512 Ultrachat calibration samples, seq 2048, sequential pipeline):
   NVFP4 W4A4 group-16 on every text projection — MLP gate/up/down (all 64
   layers), full-attention Q/K/V/O, GDN in_proj_qkv/z/out_proj — with the
   9-tensor BF16 exception pattern of the registered Qwen3.6-27B profile;
   token embedding + output head in W8G32 groupwise (matching the cometkim
   nvfp4full profile); vision tower on the official q6/q8 allocation; MTP and
   DFlash2 drafter carried in BF16.
2. Global-divisor normalization (our own tool, the key trick). llm-compressor
   emits a per-module weight_global_scale; the NInfer engine's native A4 input
   route requires each layer's fused attention parent (GDN qkvz = 16,384 rows,
   full-attn qkgv = 14,336 rows) to be one contiguous region with a single
   divisor. We unify each packing group's divisor to D = min(dᵢ) and rescale
   every member's E4M3 block scales by D/dᵢ (RNE, shrink-only → overflow-free).
   Verified dequant-invariance: worst drift 2.7% on the smallest-magnitude
   elements, 0.0000% median — weights unchanged up to E4M3 re-encode noise.
   With uniform divisors the converter coalesces each group into one fused
   parent and the engine accepts the artifact unmodified.
3. Packing & conversion with the NInfer v3 converter
   (tools/convert @ fork 3090a5b1, 2026-09-24 rebase of upstream),
   --components text,vision,mtp,dflash2 --proposal.

Artifact

Serving (RTX 5090 32 GB, single GPU)

ninfer-serve qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer \
  --host 0.0.0.0 --port 8080 --model-id Qwen3.8-27B \
  --max-context 262144 --kv-capacity auto --kv-dtype k8v4 \
  --max-concurrency 4 --default-max-tokens 32768 \
  --pending-timeout-ms 900000 --prefill-chunk 4096 \
  --temperature 0.9 --min-p 0.05 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft \
  --host-kv-mib 49152 --host-state-slots 16 \
  --max-private-continuations 16 --max-shared-prefixes 16 \
  --vision --default-thinking-budget 16384 --preserve-thinking \
  --image-token-budget 1280

Live capacity on one 5090: weights 18.0 GiB, device KV pool 308,736 tokens
(auto), k8v4, 48 GiB pinned host-KV arena, 262,144-token max context at
concurrency 4. Full context is verified working (below).

Benchmarks — same-session A/B vs the prior production artifact

Both sides served by the same engine build, same port, fresh generations,
temp 0, no cache reuse, RTX 5090 @ 450 W / SM clock pinned 2280 MHz.

Long-context recall method: needle-in-a-haystack, haystack 1,000,000 chars
(~250k tokens), secret at three depths, exact-match verdicts, zero corrupted /
one transient empty response at 75k (passed on re-run).

Why it is smaller and faster

The prior profile stored attention in heavier formats and carried a 2.23 GB
W8G32-encoded drafter. This build quantizes the entire text backbone (attention
included) to NVFP4 behind fused parents — NVFP4 tensor-core GEMMs being the
fastest path on Blackwell — and the freed VRAM is handed to the device KV pool
by --kv-capacity auto (+29,824 tokens ≈ 0.7 GiB). Abliteration itself adds
zero size: it is a rank-1 weight edit, no new parameters.

Known behaviour

• At temperature 1.0 the model occasionally emits duplicate tool calls or a
  short repetition loop in streaming tool-call tests (all emitted calls remain
  valid JSON; the streaming parser handles them). At temperature 0 this has
  never been observed across many runs; production sampling (temp 0.9,
  min-p 0.05) suppresses it.

License

Apache-2.0, inherited from the base artifact chain. Abliteration and base-model
licenses: see the credited repositories above — all credit to d0xin, ukisai,
huihui-ai, z-lab and Neroued.