← back to catalog · registered 2026-11-12 06:03

kabcnd/Huihui-Qwen3.8-27B-Abliterated-Hybrid-GGUF

kabcnd 27B GGUF second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/kabcnd%2FHuihui-Qwen3.8-27B-Abliterated-Hybrid-GGUF"
Response includes
  • classification m8
  • files 4
  • author_summary 3 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
370
↑ 14% in 90 days
Likes
2
Model age
1w ago
created 2026-11-10
Downloads over time
Now 370 → from 325 ↑ 14%
323340357375 325 on Nov 12 370 on Nov 22 370 on Nov 17 Nov
Nov 12 → Nov 22 · 11 snapshots · spans 10 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ru
Tags
gguf kenzan ternary abliterated uncensored redirect text-generation en zh ru base_model:huihui-ai/Huihui-Qwen3.8-27B-abliterated base_model:quantized:huihui-ai/Huihui-Qwen3.8-27B-abliterated

Related

Total size
13.3 GB
Files
4
Quantizations
1
Registered
2026-11-12 06:03
Last updated on HF
2026-11-18 06:19

Files by quantization

Auxiliary files 4 files 13.3 GB
Huihui-Qwen3.8-27B-Abliterated-Hybrid-PTQ1_0-Q4_1-LUT.gguf 6.86 GB 70f275b2 download
Huihui-Qwen3.8-27B-Hybrid-CoreQ4_1.gguf 6.43 GB 7ca752ce download
README.md 3.68 KB b6c4059d download
.gitattributes 1.65 KB 6e99de3d download

README current version from Hugging Face


license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
tags:

  • gguf
  • ternary
  • 1.75-bit
  • hybrid
  • abliterated
  • uncensored
  • qwen
  • apple-silicon
  • metal
  • codebook
  • speculative-decoding
  • dflash2
  • llama.cpp
    pipeline_tag: text-generation
    language:
  • en
  • zh
  • ru

☕ Support Local AI Development
Help me get a Mac mini 24GB to optimize local AI models up to 80B parameters.
Donate via Ko-fi →
Every contribution helps move the project forward.

Huihui-Qwen3.8-27B Abliterated (Hybrid PTQ1_0-LUT + CoreQ4_1)

Optimized hybrid quantization of huihui-ai/Huihui-Qwen3.8-27B-abliterated engineered for Apple Silicon (Mac Mini M6 16 GB Unified Memory, Metal GPU).


Benchmark Comparison: Current Champion vs Baseline

Metric Original FP16 Baseline Previous Hybrid v1 Our Champion (CoreQ4_1)
Model Package Size ~54.0 GiB 6.86 GiB 6.43 GiB
Average Precision 16.0 bpw 1.85 bpw 1.75 bpw (Hybrid PTQ1_0 + CoreQ4_1)
Perplexity (PPL) 3.65 3.9214 3.6742 ± 0.3901 (Native FP16 Parity!)
Speculative Speed (DFlash2) N/A 14.2 t/s 20.87 tokens/sec
Metal GPU Working Set > 54 GB (OOM) 7.1 GB 6.43 GiB (100% Resident)

Evolution History: What Was Tried & What Worked

  1. Native Ternary PTQ1_0 (6.2 GiB): Showed extreme compression efficiency, but lost slight nuance in attention output projections.
  2. Hybrid LUT + Calibrated Codebooks (6.86 GiB): Restored output projection and sensitive intermediate blocks via calibrated lookup tables. PPL recovered to 3.9214.
  3. CoreQ4_1 Splicing with DFlash2 (Current Champion, 6.43 GiB): Promoted the sensitive core blocks to Q4_1 while leaving passive weights in PTQ1_0. Paired with an asynchronous DFlash-2 draft model, achieving full FP16 perplexity parity (3.6742) and 20.87 t/s speculative generation on Apple Silicon.

🚀 Help Us Bring 80B AI Models to Affordable Hardware

Support the AI Homelab Project

I'm working on optimizing large language models to run locally on affordable hardware — without relying on expensive NVIDIA GPU servers.

My next goal is to get a Mac mini with 24GB of unified memory. This hardware upgrade will help me push local model optimization further and experiment with and optimize models of up to 80 billion parameters.

Why does this matter?

I want to explore how far we can push local AI through:

  • Advanced quantization techniques
  • Memory-efficient model architectures
  • Apple Silicon and Metal optimization
  • Faster inference with lower hardware requirements
  • Making powerful AI models more accessible to everyone

🎯 Our Next Milestone: Mac mini 24GB

Every contribution will help fund the hardware needed for further experiments, development, and optimization.

I will share benchmarks, technical findings, optimization results, and open-source work with the community.

You can help make this happen:
☕ Support the project on Ko-fi

Even a small donation makes a difference. If you cannot donate, sharing this project also helps enormously.

Thank you for supporting independent AI research and making local AI more accessible! ❤️


Quick Start (llama.cpp / llama-server)

# Run server with DFlash speculative decoding
./llama-server \
  -m Huihui-Qwen3.8-27B-Hybrid-CoreQ4_1.gguf \
  -md Huihui-Qwen3.8-27B-DFlash2-ft5-asym-Q2_K-FC-FullRank.gguf \
  --spec-type draft-dflash --spec-draft-n-max 3 --spec-draft-n-min 1 \
  --host 127.0.0.1 --port 11434 -ngl 99 -c 32768
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration