license: apache-2.0
base_model: windowsxp811203/Qwen3.8-27B-Abliterated
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- nvfp4
- fp4
- compressed-tensors
- vllm
- qwen3_5
- abliterated
- uncensored
- mtp
- qwen3.8
- qwen
- text-generation
language: - en
- zh
Qwen3.8-27B-Abliterated — NVFP4A16
4-bit NVFP4 build of windowsxp811203/Qwen3.8-27B-Abliterated,
an abliterated (refusal-removed) Qwen/Qwen3.8-27B.
55.6 GB → 28.6 GB, and the MTP draft head is intact and verified working at 76–78 % draft
acceptance. Most quantized derivatives of this architecture ship a dead MTP head, becauseQwen3_5ForConditionalGeneration.from_pretrained silently drops it and llm-compressor cannot save
a module transformers never instantiated.
Requires a Blackwell GPU (sm100+) and vLLM.
What is and isn't quantized
| group | treatment | count |
|---|---|---|
MLP gate/up/down (64 layers) + full-attention q/k/v/o (16 layers) |
NVFP4 — 4-bit float, group size 16, float8_e4m3 scales |
256 Linears |
mtp.* (draft head) |
bf16, grafted back after quantization, in ignore |
15 tensors |
model.visual.* (vision tower) |
bf16 — kept bit-identical | 167 weight tensors (333 incl. biases/norms) |
linear_attn.* (Gated DeltaNet / SSM) |
bf16 — quantization-sensitive | 336 tensors |
lm_head, embeddings |
bf16 |
The MTP tensors are both grafted back and listed in quantization_config.ignore. Both halves
matter: without the graft there is no draft head at all, and without the ignore entry vLLM's
compressed-tensors loader treats the bf16 head as a quantization target, finds no scales, and
rejects every draft — 0 % acceptance while the logs look perfectly clean.
Verification
Measured on this exact checkpoint on an RTX PRO 6000 Blackwell.
MTP speculative decoding — {"method":"mtp","num_speculative_tokens":1}:
| metric | value |
|---|---|
| Avg draft acceptance rate | 76.4 % – 78.3 % |
| Mean acceptance length | 1.76 – 1.78 |
| Accepted / drafted | 1615 / 2113 tokens |
That number is the proof the graft worked; a broken MTP head reads 0 %.
Refusal — greedy, non-thinking, no prefill jailbreak. Unmodified Qwen3.8-27B refuses
99.04 % of AdvBench under identical settings:
| benchmark | result |
|---|---|
| AdvBench (80) | 0/80 · 0.00 % |
| HarmBench safety categories (119) | 0/119 · 0.0 % |
| HarmBench copyright (41) | 17/41 · 41.5 % |
Safety categories = chemical/biological, cybercrime, harassment, harmful, illegal, misinformation
— every one exactly zero. The copyright column is not a safety refusal and is mostly
classifier false positives: the model delivers the lyrics or passage, but the text trips the
keyword list (either the generated prose itself opens with "I cannot quite…", or a pedantic
"I cannot generate a new passage … but here is a long excerpt" precedes the excerpt).
Capability — MMLU, 400 equidistant questions, identical prompting and parsing for both:
| build | MMLU |
|---|---|
| GGUF Q8_0 (reference) | 78.00 % |
| NVFP4A16 (this) | 77.75 % |
One question apart. (Do not compare these to the parent card's 82.35 %: that figure was measured
by next-token logit comparison, a different and more forgiving method. Only same-method numbers
are comparable.)
Usage
vllm serve windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4 \
--max-model-len 8192 \
--speculative-config '{"method":"mtp","num_speculative_tokens":1}'
Thinking is on by default; disable per request with"chat_template_kwargs": {"enable_thinking": false}.
1M context (verified)
The model's declared native limit is 262,144. The 1M configuration from the official Qwen3.8-27B
recipe works on this checkpoint and was measured end-to-end:
vllm serve windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4 \
--tensor-parallel-size 2 --max-model-len 1010000 \
--hf-overrides '{"text_config": {"max_position_embeddings": 1010000}}'
Needle-in-a-haystack, passphrase buried at 50 % depth, greedy:
| context | prompt tokens | retrieved | time |
|---|---|---|---|
| 1 M | 823,878 | ✅ | 364 s on 2× RTX PRO 6000 Blackwell |
Budget ≈ 61 GiB of KV at 1M (16 full-attention layers × 4 KV heads × 256 dim) on top of the
26.6 GiB of weights, so 1M needs two 96 GB cards; 256K fits comfortably on one.
If vLLM fails to start with a FlashInfer error
On hosts where the CUDA toolkit and FlashInfer's bundled CCCL headers disagree, FlashInfer's JIT
fails to build its sampling kernels and vLLM aborts withFlashInfer requires GPUs with sm75 or higher (a misleading message — the real cause is that the
capability probe itself failed). Working around it:
export VLLM_USE_FLASHINFER_SAMPLER=0
export VLLM_ATTENTION_BACKEND=TRITON_ATTN
This is a host toolchain issue, not a property of these weights.
Provenance
Quantized with llm-compressor 0.13.0 (NVFP4A16) from the bf16 parent, which was produced by
orthogonalizing 131 residual-writing tensors (including embed_tokens) against a refusal direction
at λ=1.5, leaving the vision tower byte-identical. Full recipe and evaluation in the
parent model card.
A llama.cpp build is at
Qwen3.8-27B-Abliterated-GGUF.
Support / 打賞
If these models are useful to you, tips are appreciated — they pay for the GPU time.
如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。
USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888
Disclaimer
This model will not refuse. It is published for alignment and safety research. You are responsible
for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the
base model.