language:
- en
license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.5-4B-abliterated
datasets: - f3nc3r/Qwen3.8-Flash-Next-abliterated-top64-cubits
tags: - qat
- qad
- fraqat
- hestia
- ternary
- quantization-aware-training
- qwen3_5
Huihui-Qwen3.5-4B-abliterated-QAT
This repository contains the FraQAT + Hestia Quantization-Aware Training (QAT) & Quantization-Aware Distillation (QAD) checkpoint of huihui-ai/Huihui-Qwen3.5-4B-abliterated.
- Original Student Model: huihui-ai/Huihui-Qwen3.5-4B-abliterated
- Teacher Logit / Cubit Dataset: f3nc3r/Qwen3.8-Flash-Next-abliterated-top64-cubits
- Teacher Source Model: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF
QAT / QAD Training Process (Workflow 3: FraQAT + Hestia)
- Offline Top-64 Teacher Logit ("Cubit") Extraction:
Top-64 token indices and log-probabilities were pre-extracted from ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF using allama.cppC API extractor (gguf_topk_extract.c) and packed into 104safetensorsshards (415 sequences, 844,023 token positions), hosted at f3nc3r/Qwen3.8-Flash-Next-abliterated-top64-cubits. - Sparse Vocabulary Translation Bridge:
A sparse token-mapping bridge (exact_match_ratio = 1.00) aligned the teacher and student vocabularies (248,077active tokenizer rows inside248,320padded embedding rows). - Hestia Differentiable Softmax-STE Ternary QAT (
step3_fraqat_core.py):- Scope & Sparing Policy: 216 internal linear modules (76.884% of 4.21B parameters:
3,233,546,240/4,205,751,296) were wrapped inHestiaTernaryLinear({-1, 0, +1}ternary grid with learnablefp32per-output-row scales). Sensitive layers—embed_tokens,lm_head,*norm*,self_attn.o_proj, andlinear_attn.out_proj—were spared in BF16. - Temperature Annealing Schedule: 500 total optimizer steps (
adamw8bit,lr=2e-5) on an NVIDIA H200: 10% warmup, 75% cosine temperature annealing (tau: 1.0 -> 0.0), and 15% hard-ternary scale fine-tuning with frozen latent ternary weights. - Joint QAD Objective:
$$\mathcal{L} = (1 - \alpha)\mathcal{L}{\text{CE}} + \alpha\mathcal{L}{\text{KL}} \quad (\alpha = 0.7)$$
combining next-token cross-entropy with chunked bridged KL divergence against the teacher's top-64 logit cubits.
- Scope & Sparing Policy: 216 internal linear modules (76.884% of 4.21B parameters:
- Unwrapping & Verification:
HardenedHestiaTernaryLinearlayers were unwrapped back into standardnn.Linearmodules (model.safetensors, zero leaked scale keys) and verified withAutoModelForCausalLM.
Verified Quality Metrics (quality_report.json)
| Stage | Cross-Entropy (Shard 0 Probe) |
|---|---|
| Stock BF16 Student | 2.6216 |
| Untrained Hard-Ternary Wrap | 18.3398 |
| Post-Hardening (500 steps) | 2.0683 (quality_status: ok) |
Included Files
model.safetensors— Unwrapped QAT-hardened BF16-container weights (76.884%ternary-grid linear layers)Huihui-Qwen3.5-4B-ab_QAT-BF16.gguf— Full-size BF16 GGUF exportHuihui-Qwen3.5-4B-ab_QAT-Q3_K_M.gguf—Q3_K_MGGUF baseline exportconfig.json,generation_config.json,tokenizer.json,tokenizer_config.json,chat_template.jinjaquality_report.json,train_fraqat_w3.log