library_name: mlx
tags:
- mlx
- oq
- quantized
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-oQ6e_g128-fp16-text
This model was quantized using oQ (oMLX v0.7.0.dev4) mixed-precision quantization.
Quantization details
- Model type: qwen3_5
- Bits: 6
- Group size: 128
- Format: MLX safetensors
"Done. Perplexity + KLD fidelity, all four on wikitext-2-raw (51,100 positions, top-512 support, fp32 logits) vs your oQ8e-mtp standard:
variant perplexity dPer% fwdKLD revKLD JSD top1
oQ8e-mtp (base) 7.9190 0 - - - 1.0000
oQ6e-mtp 7.9215 +0.032 0.0013 10.62 5.31 0.9839
oQ6e-g128-text 7.9233 +0.055 0.0017 10.88 5.44 0.9805
oQ63e-text 7.9401 +0.267 0.0031 11.78 5.89 0.9764
Read: lower Δperp, lower fwd KLD / JSD, higher top-1 = closer to your fp16-adjacent reference.
The gist:
- oQ6e-mtp is the best of the three — nearly indistinguishable from base (Δperp 0.032%). MTP-headed, ~5.6s/window.
- oQ6e-g128-text is a hair behind (Δperp 0.055%). Surprisingly, the wider g128 group holds up basically at parity — and it's the fastest (4.7s/window, no MTP head) — so it's the best size/speed tradeoff.
- oQ63e-text is the worst: Δperp 0.27%, roughly 8-10× the divergence of the two oQ6 builds. Still usable, but it's distinctly the lowest-fidelity.
One thing to note: the -mtp tag is nominal here — all four configs report mtp_layers:[], so the model output is a single (1, seq, 248320) array and my comparison used the main-LM logits throughout. No MTP-head bias to correct.
The KLD asymmetry is the usual oQ signature: fwd KLD stays tiny (0.001–0.003 nat) because the reference covers what the quant agrees on; reverse KLD is large (10.6–11.8) because quantization concentrates probability on a few tokens the fp16 reference barely registers. Perplexity and top-1 divergences tell the practical story — all three are comfortably deployable, but if you want the closest-to-standard quality, take oQ6e-mtp; if you want speed at negligible cost, oQ6e-g128-text.
Artifacts: /Users/juha/.hermes/cache/scratch/kld_results/*.json (full per-variant numbers), kld_ref.npz + wikitext_windows.npy (reusable reference). Skill updated with the rebuilt harness entry points."