license: apache-2.0
base_model: DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
  tags:
• nvfp4
• fp4
• w4a4
• gptq
• quantized
• compressed-tensors
• llm-compressor
• vllm
• vision-language
• thinking
• mtp
• speculative-decoding
• math-calibration
• gsm8k
• kv-cache-calibrated

Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882 — NVFP4 (W4A4, KV-calibrated)

Quantization-only derivative of
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
(the 735-882 "Stage 1b" Heretic checkpoint). No training, no additional
abliteration. All credit for the model itself belongs upstream.

A note on the additional math calibration data

A first attempt calibrated only on a (reasoning + chat + code) blend, and
its grade-school math collapsed sharply. Measured on identical hardware (RTX
PRO 6000 Blackwell, lm-eval 0.4.12 vLLM backend, greedy, 0-shot raw prompts):

Adding math to the calibration blend lifts gsm8k by roughly +0.09 over the
no-math build, with no other task regressing (see Quality Verification). The
residual gap to the BF16 source (0.600 vs 0.899) is, we think, the expected
W4A4 cost on a reasoning-heavy task. Doubling the math calibration in the
shipped build was also tried; it did not measure better than the shipped
build, so the calibration above is the one shipped.

Quality Verification

All values below were benchmarked under identical conditions:
lm-eval 0.4.12 (vLLM backend) in the vllm/vllm-openai:qwen38-x86_64-cu130
image (torch 2.13.0+cu130), single RTX PRO 6000 Blackwell (96 GB),
--gpu-memory-utilization 0.85, max_model_len 4096, greedy decoding.
Prompts are 0-shot raw task prompts — no chat template, no thinking (the
lm-eval runs recorded chat_template: null; equivalent to the "Instruct mode"
the upstream card benches in).

* gsm8k is the largest single W4A4 cost on this base; the
math-calibration section above covers it.

Reading: on every other task the W4A4 build is within ~3 pt of the BF16
source. Absolute values differ from third-party harness numbers (e.g. the
upstream card's llama.cpp benches); compare against the BF16 column, which was
measured under the same conditions.

Reproducibility

Built with llm-compressor oneshot + GPTQModifier
(recipe.yaml included: block size 128, dampening 0.01, actorder static,
MSE observer, group size 16) with a domain-matched calibration blend
(long reasoning + general chat + code + GSM8K train math), on a GB10
(sm_121, CUDA 13.0, torch 2.11.0+cu130, transformers 5.14.1, llm-compressor
0.13.0), with the KV-cache calibration pass enabled (kv_cache_scheme:
fp8, symmetric, per-tensor, memoryless_minmax) riding on the same one-shot
pass.

What the KV calibration adds. 32 scalar FP32 tensors —
k_scale/v_scale (one per full-attention layer's k_proj/v_proj;
this model has 16 full-attention layers, the rest being Gated DeltaNet
linear attention, which keeps a recurrent state and has no paged KV cache)
— plus quantization_config.kv_cache_scheme in config.json.
vLLM's compressed-tensors path reads these into the attention layers
and multiplies the loaded K/V entries by the per-layer scale when
quantizing them into the fp8 paged cache.

Post-quantization structural fixes (MTP restoration, index.json
rebuild, ignore guarantee for mtp.* in config.json, tower split into
model-towers.safetensors) are idempotent and re-derivable from the BF16
source. Structural check: PASS.

Serve with vLLM

vllm serve <this-repo> \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --kv-cache-dtype fp8 --enable-prefix-caching \
  --max-num-batched-tokens 2048 --max-num-seqs 2 \
  --gpu-memory-utilization 0.44 \
  --reasoning-parser qwen3 --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

(Flag values are a tested GB10 / 128 GB unified-memory configuration —
256K context works at this setting. On a dedicated GB10 you can raise
--gpu-memory-utilization toward ~0.8 for a larger KV pool; avoid ~0.95,
which starves the OS on a unified pool.)

Engineering notes

• This quantization run (calibration, QC, card, upload) was performed with
  an LLM agent operating the operator's own toolchain.

Lineage

Qwen/Qwen3.8-27B → DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU → this.
Apache 2.0, unbroken from the base model.