license: apache-2.0
base_model: DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
  tags:
• nvfp4
• fp4
• w4a4
• gptq
• quantized
• compressed-tensors
• llm-compressor
• vllm
• vision-language
• thinking
• mtp
• speculative-decoding
• math-calibration
• gsm8k
• kv-cache-calibrated
• chat-template-patched

Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L — NVFP4 (W4A4, KV-calibrated)

Quantization-only derivative of
DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored
(the 709-L "Stage 2b-rplus3" TWIN-TURBO checkpoint). No training, no
additional abliteration. All credit for the model itself belongs upstream.
One exception is the chat template, which is patched in this repo —
see Chat template (patched in this repo).

Of the Qwen3.8-27B DavidAU derivatives we have published, this 709-L base
measured out as the one we would recommend first: it is the strongest of
the three on raw benchmarks, and the W4A4 cost here is the smallest we
have seen on this architecture.

This is a vLLM-first build: the quant format, the KV-cache calibration,
the MTP speculative config, and the chat-template patch below are all
tuned for vLLM's compressed-tensors path. We have not tested
llama.cpp / SGLang (and the einstein/spoon modes in particular are
unverified outside vLLM).

For Blackwell users: 4-bit activations (the "A4" in W4A4) is the
part a W4A16 build leaves at full precision, so on sm_120/sm_121-class
hardware the 4-bit activation path is the one that unlocks the extra
decode throughput — expect this build to run noticeably faster than a
W4A16 quant of the same model at the quality measured below.

Quality Verification

All values below were benchmarked under identical conditions:
lm-eval 0.4.12 (vLLM backend) in the vllm/vllm-openai:qwen38-x86_64-cu130
image, single RTX PRO 6000 Blackwell (96 GB),
--gpu-memory-utilization 0.85, max_model_len 4096, greedy decoding.
Prompts are 0-shot raw task prompts — no chat template, no thinking
(equivalent to the "Instruct mode" the upstream card benches in).
Worst relative degradation: 2.6 % (QC threshold 10 %).

Reading: every task is within ~2.5 pt of the BF16 source; gsm8k keeps
98 % of the source score. The only notable W4A4 cost is the wikitext PPL
(+8.1 %), the expected 4-bit language cost on this base. Absolute values
differ from third-party harness numbers (e.g. the upstream card's llama.cpp
benches); compare against the BF16 column, which was measured under the
same conditions.

Chat template (patched in this repo)

This repo ships a hardened chat_template.jinja as the default, and
the unmodified upstream template as chat_template-upstream-original.jinja
for reference. The exact patch is in template_patch/chat_template.hardened.diff.

Upstream base: the template as of commit 9cdc928b of the upstream
repo (2026-09-12, the agentic/tool-calling fix). That fix is folded into
this file; everything else in it is upstream.

Potential risks in the upstream template as shipped. It implements an
in-message mode switch by scanning messages for a literal {REASON: tag.
For many uses this is never a problem, but the design carries several
independent sharp edges:

• Scope: it scans every message in the history — assistant, tool
  results, everything — not just what the user typed. Any text that merely
  quotes the feature (reading a file, fetching a URL, a RAG document, a
  pasted example) can silently switch the model's reasoning mode, or fail
  the request entirely.
• Hard failure: a value outside the whitelist (xhigh/medium/low/ einstein/spoon) makes the template raise_exception, aborting the whole
  request — instead of being ignored.
• Self-trigger: the template source itself contains the literal
  {REASON:, so the moment the file's own content appears in the
  conversation (e.g. an agent that inspects its own config), the parser
  extracts a code fragment as the "mode" and every later request in
  that session fails until a new chat is started.
• Vision path: the scan pass renders every message as if it were a
  system message, so a user message containing an image or video
  fails the whole request with System message cannot contain images
  before generation even starts. In other words, served via vLLM, image
  input was not usable at all with this template. We think the upstream
  GGUF card's "Vision: tested" refers to the GGUF/mmproj path (llama.cpp),
  which does not run this jinja scan and never hit this code path — so
  the issue was not visible from the GGUF side.

The patch closes all four of these with a 7-line change to the parse
block; every other line is byte-identical to upstream.

Semantics that differ from the upstream README. The mode switch is now
deliberately conservative and stateless:

The five modes themselves and their system-prompt blocks are untouched.
Verification: 24/24 robustness/repro checks (including both incident
repros on the unmodified template) + byte-for-byte output parity with the
original template for all 5 thinking + 5 instruct modes (local jinja2;
same engine family as vLLM).

API note. The OpenAI-compatible top-level reasoning_effort field is
still validated by vLLM against the standard 7-value ladder, so
reasoning_effort="einstein"/"spoon" at the API level is rejected
regardless of template. Use extra_body={"chat_template_kwargs": {"reasoning_effort": "einstein"}}, or the in-message tag.

If you prefer the exact upstream template, serve with
--chat-template <repo>/chat_template-upstream-original.jinja.

Reproducibility

Built with llm-compressor oneshot + GPTQModifier
(recipe.yaml included: block size 128, dampening 0.01, actorder static,
MSE observer, group size 16) with a domain-matched calibration blend
(long reasoning + general chat + code + GSM8K train math), on a GB10
(sm_121, CUDA 13.0, torch 2.11.0+cu130, transformers 5.14.1, llm-compressor
0.13.0), with the KV-cache calibration pass enabled (kv_cache_scheme:
fp8, symmetric, per-tensor, memoryless_minmax) riding on the same one-shot
pass.

What the KV calibration adds. 32 scalar FP32 tensors —
k_scale/v_scale (one per full-attention layer's k_proj/v_proj;
this model has 16 full-attention layers, the rest being Gated DeltaNet
linear attention, which keeps a recurrent state and has no paged KV cache)
— plus quantization_config.kv_cache_scheme in config.json.
vLLM's compressed-tensors path reads these into the attention layers
and multiplies the loaded K/V entries by the per-layer scale when
quantizing them into the fp8 paged cache.

Post-quantization structural fixes (MTP restoration from the source's
model-mtp-restored.safetensors, index.json rebuild, ignore
guarantee for mtp.* in config.json, tower split into
model-towers.safetensors) are idempotent and re-derivable from the BF16
source. Structural check: PASS (source keys 1199, missing 0).
MTP draft head verified working with the speculative config below;
vision tower verified structurally (weights intact).

Serve with vLLM

vllm serve <this-repo> \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --kv-cache-dtype fp8 --enable-prefix-caching \
  --max-num-batched-tokens 2048 --max-num-seqs 2 \
  --gpu-memory-utilization 0.44 \
  --reasoning-parser qwen3 --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

(Flag values are a tested GB10 / 128 GB unified-memory configuration —
256K context works at this setting. On a dedicated GB10 you can raise
--gpu-memory-utilization toward ~0.8 for a larger KV pool; avoid ~0.95,
which starves the OS on a unified pool.)

Engineering notes

• This quantization run (calibration, QC, card, upload) was performed with
  an LLM agent operating the operator's own toolchain.
• The chat template hardening (incident repro, patch, and test suites) was
  likewise done with the operator's agent; the diff is in template_patch/.

Lineage

Qwen/Qwen3.8-27B → DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
→ DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored → this.
Apache 2.0, unbroken from the base model.