language:
- en
license: apache-2.0
tags: - llama.cpp
- gguf
- quantization
- sub-2bpw
- recipe-b
- miniquant
- qwen35
- qad
base_model: DavidAU/Qwen3.8-27B-Ultimate-Heretic-Uncensored
Qwen3.8-27B-Ultimate-Heretic-Uncensored-miniQuant (Recipe B)
Sub-2.0 BPW miniQuant Recipe B GGUF quantization of DavidAU/Qwen3.8-27B-Ultimate-Heretic-Uncensored, built and served with fencerJP/llama.cpp-miniQuant.
Available Checkpoints & Benchmark Results
| File | Recipe / Variant | Size | Effective BPW | Prefill Speed (CPU) | Decode Speed (CPU) | Quality |
|---|---|---|---|---|---|---|
DavidAU-Qwen3.8-27B-Ultimate-RecipeB-Distilled.gguf |
Recipe_B (IQ1_S + IQ2_K anchors + imatrix + LMS) |
5.84 GiB (5,980.66 MB) | 1.862 bpw | 29.3 – 47.1 tok/s | 7.17 tok/s | Verified exact answers (15+27=42), < 2.0 bpw |
DavidAU-Qwen3.8-27B-Ultimate-Sub2Bpw-Direct.gguf |
Recipe_B Direct (IQ1_S + IQ2_K anchors) |
5.81 GiB (5,950.12 MB) | 1.852 bpw | 36.9 – 47.1 tok/s | 5.10 tok/s | 75.0% Coherence, 66.7% Accuracy (100% factual) |
Creation Process (QAD to Recipe B)
These checkpoints were created using the miniQuant Recipe B sub-2.0 bpw pipeline:
- QAT / QAD Foundation: Starting from Quantization-Aware Training / Distillation (FraQAT + HESTIA bridged-KL distillation) checkpoints in BF16.
- Commutative Rotation Partitioning: 256-dim paired Hadamard rotations applied strictly to contraction down-projections (
ffn_down,ssm_out,attn_output), preserving Gated DeltaNet recurrent state projections (ssm_in,conv1d,dt,alpha,beta) and embeddings in their natural coordinate basis. - Non-Uniform Multi-Bit Sparing (
llama-quantize --recipe-b):- Bulk Trunk (
ffn_up,ffn_gate,attn_qkv,ssm_in):IQ1_S(~1.56 bpw) - FFN Down Projections (
ffn_down):IQ2_XXS(~2.06 bpw) - Attention/SSM Outputs & Values (
attn_output,ssm_out,attn_v):IQ3_XXS(~3.06 bpw) - Boundary Anchors (
token_embd.weight,output.weight):IQ2_Kon 27B/40B (where embeddings are <5% of weights), orIQ4_Kon 4B (where the 248k-vocab embedding table is 15.1% of weights).
- Bulk Trunk (
- Closed-Form LMS Scale Distillation: Out-of-core Least Mean Squares block scale ($d$) refinement against the unquantized BF16 teacher weights.
Hardware Specifications (Creation & Benchmarking)
- CPU: AMD Ryzen AI 9 HX 470 w/ Radeon 890M (12 Cores / 24 Threads, AVX-512 / AVX-VNNI / AVX-512 BF16)
- System RAM: 121 GiB DDR5
- GPU: NVIDIA Tesla V100-SXM2-16GB (16 GiB VRAM)
- Inference / Quantization Engine: fencerJP/llama.cpp-miniQuant (
master/recipe-b)
Usage (llama.cpp-miniQuant)
./build/bin/llama-cli \
-m DavidAU-Qwen3.8-27B-Ultimate-RecipeB-Distilled.gguf \
-p "<|im_start|>user\nWhat is 15 plus 27? Answer with the exact number.<|im_end|>\n<|im_start|>assistant\n" \
-n 128 -t 16 --temp 0.0 --repeat-penalty 1.15