library_name: mlx
license: other
license_name: qwen-community-1.0
license_link: LICENSE
base_model: huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
base_model_relation: quantized
pipeline_tag: text-generation
tags:
- mlx
- safetensors
- oq
- quantized
- mtp
- mixed-precision
- abliterated
Huihui-Qwen3.8-Flash-Next-abliterated-v2-oq4-mix6-mtp
⏳ 上傳中 / Uploading — model weights are still transferring; final file
list verification follows on completion.
A U-shaped oQ4 mixed-precision MLX quantization of
huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
(unsloth-Q8_0). -mtp: MTP drafter included, spec-decoding ready.
Weights re-quantized from the Q8_0 GGUF source under the mix6 per-tensor
override recipe; GDN linear-attention tensors de-interleaved (GGUF tiled head
order → native grouped order) before quantization. MTP drafter and n-gram
(PLE) tensors carried from the mix6-mtp base.
Recipe
Default bits=6, group_size=64, mode=affine; per-family overrides:
| Component | Bits / group |
|---|---|
| GDN projections + full-attention q/k/v/o/indexer (edge & sensitivity layers) | 8-bit g64 |
| Shared experts (48 layers) | 8-bit g128 |
| Hyper-connections (attn/mlp inject) | 8-bit g64 |
| MoE routed experts, middle 40 layers (L4–43) | 4-bit g64 (120 explicit keys) |
| MoE routed experts, edge 8 layers (L0–3, L44–47) | 6-bit g64 — U-shaped edges (default) |
| N-gram (PLE) table shards | 6-bit g32 |
| MTP drafter | 6/8-bit (from mix6-mtp base) |
| Norms / routers / vision tower | bf16, unquantized |
Quantization config identical (993/993 keys) to the ThinkingCap mix6-v2
sibling. Verified key-by-key at dequant level against that oracle: all 686
quantized tensors within quantization noise (max rms 0.047).
Usage
model, processor = load("umixer/Huihui-Qwen3.8-Flash-Next-abliterated-v2-oq4-mix6-mtp")
License & provenance
License follows the base model (Qwen/Qwen3.8-Flash-Next, license: other).
Chat template from the huihui GGUF (Unsloth-aligned).