Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored (W8A8 Mixed)
Mixed-precision quantized version of Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored.
Quantization
| Aspect | Detail |
|---|---|
| Method | compressed-tensors (Unsloth 2026.7.2) |
| Format | mixed-precision |
| Source precision | bfloat16 (~55.6 GB, 12 shards) |
| Quantized size | ~25.1 GB (8 shards + 0.85 GB extra tensors) |
| Size reduction | ~55% |
Precision groups
| Group | Weight bits | Activation bits | Strategy | Layers | Format |
|---|---|---|---|---|---|
| W8A16 | 8 (int, symmetric) | BF16 (unquantized) | channel, static | 66 | pack-quantized |
| W8A8 | 8 (int, symmetric) | 8 (int, symmetric) | channel + token (dynamic) | 143 | int-quantized |
| W4A16 | 4 (int, symmetric) | BF16 (unquantized) | group (size 128), static | 191 | pack-quantized |
Layers kept at full BF16 precision (excluded from quantization): all 26 vision blocks, all linear-attention projection layers (in_proj_a, in_proj_b, norm, linear_attn), and lm_head. These are stored in model_extra_tensors.safetensors.
Architecture
- Model type:
qwen3_5(Qwen3_5ForConditionalGeneration — multimodal, text + vision) - Hidden size: 5120
- Layers: 64 (linear-attention with every 4th layer being full attention)
- Attention heads: 24 Q / 4 KV (full); 16 key / 48 value (linear)
- Head dim: 256 (full) / 128 (linear)
- Context: 262 144 tokens
- Vocab size: 248 320
- Parameters: ~13.4 B (post-quantization weight storage: 24.25 GB)
Benchmark
AIME 2026 (both AIME I and II)
| Metric | W8A8 (this repo) | FP16 baseline |
|---|---|---|
| Accuracy | 90.0% (27/30) | 86.7% (26/30) |
| Problems | 30 (15 AIME I + 15 AIME II) | 30 (15 AIME I + 15 AIME II) |
| Protocol | 0-shot, boxed numeric answer | 0-shot, boxed numeric answer |
| Avg latency | 216.7 s | 356.3 s |
| Avg output tokens | 16 625 | 15 590 |
| Total tokens (run) | 31 003 input / 498 739 output | 31 003 input / 467 694 output |
| Parallelism | 8 concurrent requests | 8 concurrent requests |
Run date: 2026-09-19 (both runs).
- W8A8 (this repo): one problem hit the 100K max-tokens cap (likely a false negative from truncated reasoning).
- FP16 baseline: 4 problems produced no
\boxed{}answer (reasoning ran to the token cap without a final answer); the remaining 26 were correct. Within run-to-run noise on a 30-problem set, the quantized W8A8 scores on par with (slightly above) full FP16.
Usage
Load with SGLang or vLLM using the quantization config in this directory:
# SGLang
sglang.launch_server --model-path /path/to/this-dir --port 8002
# vLLM (if supported by your vLLM version)
vllm serve /path/to/this-dir --port 8002
The quantization_config.json is automatically detected by runtimes that support the compressed-tensors format. No additional flags are needed for the mixed-precision scheme.
Source model
- DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored
- Stage2b-rplus3 (internal name), ULTRA Heretic variant (6/100 refusals, KL 0.0397)
- 5 reasoning / 5 instruct modes, switchable on the fly