← back to catalog · registered 2026-11-13 02:03

f3nc3r/Huihui-Qwen3.5-4B-abliterated-QAT

f3nc3r 4B GGUF second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/f3nc3r%2FHuihui-Qwen3.5-4B-abliterated-QAT"
Response includes
  • classification m8
  • files 12
  • author_summary 4 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
2K
Likes
1
Model age
1w ago
created 2026-11-13

Training datasets

Corpora the author lists in the model card. Datasets tracked in our /datasets catalog carry a category badge linking to the workflow stage. Others open on Hugging Face.

Downloads over time
Now 1.9K → from 0 ↑ 0%
07051.4K2.1K 0 on Nov 13 1.9K on Nov 22 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en
Quantizations
BF16 Q3_K
Tags
safetensors gguf qwen3_5_text qat qad fraqat hestia ternary quantization-aware-training qwen3_5 en dataset:f3nc3r/Qwen3.8-Flash-Next-abliterated-top64-cubits

Related

Total size
17.8 GB
Files
12
Quantizations
3
Registered
2026-11-13 02:03
Last updated on HF
2026-11-13 01:12

Files by quantization

BF16 1 file 7.85 GB
Huihui-Qwen3.5-4B-ab_QAT-BF16.gguf 7.85 GB 19f3b5ad download
Q3_K 1 file 2.11 GB
Huihui-Qwen3.5-4B-ab_QAT-Q3_K_M.gguf 2.11 GB aad308f1 download
Auxiliary files 10 files 7.85 GB
model.safetensors 7.83 GB 62bf5187 download
tokenizer.json 19.1 MB a5cd9732 download
train_fraqat_w3.log 22.0 KB c84e4155 download
chat_template.jinja 7.57 KB a585dec8 download
README.md 3.61 KB 1df3da74 download
config.json 1.90 KB 98bcbb16 download
.gitattributes 1.67 KB 12ad40bf download
tokenizer_config.json 1.16 KB 848c7c7f download
quality_report.json 482 B f8cf469a download
generation_config.json 116 B 2e978cff download

README current version from Hugging Face


language:

  • en
    license: apache-2.0
    base_model: huihui-ai/Huihui-Qwen3.5-4B-abliterated
    datasets:
  • f3nc3r/Qwen3.8-Flash-Next-abliterated-top64-cubits
    tags:
  • qat
  • qad
  • fraqat
  • hestia
  • ternary
  • quantization-aware-training
  • qwen3_5

Huihui-Qwen3.5-4B-abliterated-QAT

This repository contains the FraQAT + Hestia Quantization-Aware Training (QAT) & Quantization-Aware Distillation (QAD) checkpoint of huihui-ai/Huihui-Qwen3.5-4B-abliterated.


QAT / QAD Training Process (Workflow 3: FraQAT + Hestia)

  1. Offline Top-64 Teacher Logit ("Cubit") Extraction:
    Top-64 token indices and log-probabilities were pre-extracted from ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF using a llama.cpp C API extractor (gguf_topk_extract.c) and packed into 104 safetensors shards (415 sequences, 844,023 token positions), hosted at f3nc3r/Qwen3.8-Flash-Next-abliterated-top64-cubits.
  2. Sparse Vocabulary Translation Bridge:
    A sparse token-mapping bridge (exact_match_ratio = 1.00) aligned the teacher and student vocabularies (248,077 active tokenizer rows inside 248,320 padded embedding rows).
  3. Hestia Differentiable Softmax-STE Ternary QAT (step3_fraqat_core.py):
    • Scope & Sparing Policy: 216 internal linear modules (76.884% of 4.21B parameters: 3,233,546,240 / 4,205,751,296) were wrapped in HestiaTernaryLinear ({-1, 0, +1} ternary grid with learnable fp32 per-output-row scales). Sensitive layers—embed_tokens, lm_head, *norm*, self_attn.o_proj, and linear_attn.out_proj—were spared in BF16.
    • Temperature Annealing Schedule: 500 total optimizer steps (adamw8bit, lr=2e-5) on an NVIDIA H200: 10% warmup, 75% cosine temperature annealing (tau: 1.0 -> 0.0), and 15% hard-ternary scale fine-tuning with frozen latent ternary weights.
    • Joint QAD Objective:
      $$\mathcal{L} = (1 - \alpha)\mathcal{L}{\text{CE}} + \alpha\mathcal{L}{\text{KL}} \quad (\alpha = 0.7)$$
      combining next-token cross-entropy with chunked bridged KL divergence against the teacher's top-64 logit cubits.
  4. Unwrapping & Verification:
    Hardened HestiaTernaryLinear layers were unwrapped back into standard nn.Linear modules (model.safetensors, zero leaked scale keys) and verified with AutoModelForCausalLM.

Verified Quality Metrics (quality_report.json)

Stage Cross-Entropy (Shard 0 Probe)
Stock BF16 Student 2.6216
Untrained Hard-Ternary Wrap 18.3398
Post-Hardening (500 steps) 2.0683 (quality_status: ok)

Included Files

  • model.safetensors — Unwrapped QAT-hardened BF16-container weights (76.884% ternary-grid linear layers)
  • Huihui-Qwen3.5-4B-ab_QAT-BF16.gguf — Full-size BF16 GGUF export
  • Huihui-Qwen3.5-4B-ab_QAT-Q3_K_M.gguf — Q3_K_M GGUF baseline export
  • config.json, generation_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja
  • quality_report.json, train_fraqat_w3.log

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-13 Upload folder using huggingface_hub 3c3f59b 3.6 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration