license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: image-text-to-text
base_model: - zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated
base_model_relation: quantized
tags: - nvfp4
- fp4
- modelopt
- tensorrt
- sglang
- dflash2
- speculative-decoding
- roleplay
- creative-writing
- abliterated
- uncensored
- qwen3
- not-for-all-audiences
Qwen3.8-27B-Dominatrix-abliterated — MTP-NVFP4 (DFlash 2 compatible)
Mixed-precision NVFP4/FP8 PTQ of
zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated
— allura-org's Dominatrix roleplay finetune with huihui-ai's refusal direction projected out.
23 GB, built with NVIDIA TensorRT Model Optimizer for Blackwell (SM120) inference in
SGLang or vLLM.
The distinguishing feature: lm_head is left dense BF16, which is a hard requirement for
DFlash 2 speculative decoding. Most ModelOpt NVFP4 exports of this architecture quantizelm_head and therefore cannot run DFlash 2 at all. Cost of the dense head is ~1.9 GB of
VRAM over a packed one.
Quantization layout
| component | precision |
|---|---|
MLP gate_proj / up_proj / down_proj |
NVFP4 W4A4 |
self_attn q/k/v/o, linear_attn projections |
FP8 e4m3 |
| KV cache | FP8 |
lm_head |
BF16, dense |
embed_tokens, MTP head, vision tower |
BF16 |
Export format is ModelOpt MIXED_PRECISION with a per-layer map in hf_quant_config.json.
Calibrated on in-domain ChatML roleplay text rather than a generic news corpus.
hf_quant_config.jsonrecordsproducer.version: 0.0.0because it was built from an
editable install. That field is not meaningful provenance.
Serving
SGLang with DFlash 2
Requires the z-lab/Qwen3.8-27B-DFlash2
drafter and an SGLang build including PR #35371.
sglang serve \
--trust-remote-code \
--model-path /models/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4 \
--mem-fraction-static 0.70 \
--attention-backend flashinfer \
--chunked-prefill-size 2048 \
--reasoning-parser qwen3 \
--speculative-algorithm DFLASH \
--speculative-draft-model-path /models/Qwen3.8-27B-DFlash2-zlab \
--speculative-dflash-block-size 8 \
--speculative-draft-model-quantization unquant
--kv-cache-dtype can be omitted: this checkpoint declares kv_cache_quant_algo: FP8, so
SGLang's auto resolves it.
Without speculative decoding
Drop the four --speculative-* flags. The dense lm_head buys nothing in that configuration
but is otherwise harmless.
MTP
The mtp.* tensors survive the quant in BF16, so MTP speculation remains available as an
alternative drafter. Pick one — MTP or DFlash 2, not both.
Hardware
NVFP4 requires Blackwell (SM120+) for native FP4 tensor-core execution. Built and tested
on an RTX PRO 6000 Blackwell. Weights are ~23 GB, leaving room for a long-context KV cache and
the 2B DFlash 2 drafter.
Quality
Fidelity of the underlying BF16 abliteration versus stock Dominatrix is summarised on the
BF16 card.
The quantization error of this build has not been measured. Treat it as unquantified.
Leaving lm_head dense should help, since the output projection is among the most
quantization-sensitive layers, but that is reasoning, not a measurement.
Sampler guidance from upstream Dominatrix carries over: temperature 1.0–1.25 with min_p 0.1 or top_p 0.95; some prefer 0.7 and nothing else.