license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
tags:
- kenzan
- gguf
- ternary
- ptq1_0
- q4_1
- 1.75-bit
- hybrid
- abliterated
- uncensored
- qwen
- apple-silicon
- metal
- codebook
- speculative-decoding
- dflash2
- kenzan-router
- llama.cpp
pipeline_tag: text-generation
language: - en
- zh
- ru
inference: false
extra_gated_heading: "Mandatory Liability Waiver & Uncensored Model Agreement"
extra_gated_prompt: >
WARNING: This repository hosts an ABLITERATED (uncensored) 27B language model with safety refusals removed.
By requesting access to, downloading, or using this model (or any other models published by kabcnd / the Kenzan Project),
you explicitly acknowledge, sign, and agree to the following legally binding terms:
(1) SOLE RESPONSIBILITY: You assume 100% personal and legal responsibility for all prompts, outputs, deployments, and consequences arising from the use of this model or any other models provided by kabcnd / the Kenzan Project.
(2) FULL RELEASE OF LIABILITY: The author(s), maintainer(s), and contributors (kabcnd / Kenzan Project) bear ZERO liability and are hereby completely released and held harmless from any claims, damages, misuse, illegal acts, or direct/indirect harm caused by this model or any other Kenzan / Huihui / Flint models.
(3) LAWFUL USE ONLY: You will use this model strictly for lawful research, red-teaming, security evaluation, or authorized local testing in compliance with all applicable laws in your jurisdiction.
extra_gated_fields:
Full Name or Signature (required to sign waiver): text
Organization / Affiliation (or Independent Researcher): text
I confirm that I am at least 18 years old: checkbox
I accept 100% personal responsibility and release kabcnd / Kenzan Project from all liability for any harm caused by this or other models: checkbox
extra_gated_button_content: "Sign Liability Waiver & Unlock Download"
☕ Support Local AI Development: Help me acquire a Mac mini 24GB/32GB to optimize and test local MoE models up to 80B parameters. Donate via Ko-fi →
[!CAUTION]
Kenzan-Qwen3.8-27B-Abliterated-Core16 — "27B Uncensored, 16 Surgical Layers, 93.75% Speculative DAR, 6.43 GiB."
Abliterated models normally shatter speculative drafting at 35% acceptance — so we reverse-engineered Apple Metal'smc_c2/mc_c3GEMV kernels, restored layers 22–29 back toPTQ1_0, and pinned only the 16 true refusal-core layers (30..45) inQ4_1to hit 20.87 t/s at 128k context inside 16 GB RAM.
Kenzan-Qwen3.8-27B-Abliterated-Core16 (6.43 GiB GGUF)
Kenzan-Qwen3.8-27B-Abliterated-Core16 (Kenzan-Qwen3.8-27B-Abliterated-Core16-Q4_1-M6-16G.gguf) is a surgically repacked hybrid quantization of huihui-ai/Huihui-Qwen3.8-27B-abliterated (27B parameters, 64 layers, GQA 32/16, 128k native context) engineered for Apple Silicon 16 GB Unified Memory (Mac mini M6).
Short Summary:
Kenzan-Qwen3.8-27B-Abliterated-Core16 (6.43 GiB GGUF) — Surgical Hybrid PTQ1_0 (1.75 bpw) + Core16 Q4_1 (layers 30–45) 27B uncensored model tuned for Apple Silicon 16 GB Unified Memory, achieving FP16 PPL parity (3.6742), 93.75% DFlash2 DAR, and 20.87 t/s at 128k context.
Architecture & Surgical Core16 Pinning
When abliteration removes the rank-1 refusal direction from PTQ1_0 (1.75 bpw Codebook-LUT) weights, naive builds promote all 54 layers (22..52) to Q4_1. This inflates the model to 6.86 GiB and introduces phase drift across layers 24..30 where the DFlash2 speculative draft head taps hidden states — crashing draft acceptance rate (DAR) to 35.0% (6.9–8.05 t/s).
By measuring layer-wise Frobenius norm delta ||W_new - W_old||_F:
- Layers
0..21&46..63:0.0%refusal energy → kept in nativePTQ1_0. - Layers
22..29: Only14.8%refusal energy, but directly in the DFlash2 tap zone → restored to nativePTQ1_0. - Layers
30..45(Core16):85.2%of total refusal energy → pinned inQ4_1. - Release Metadata ID:
Kenzan-Qwen3.8-27B-Abliterated [Core16: L30-45 Q4_1 + PTQ1_0 | DFlash2 | Apple M6 16GB]
Verified Benchmarks (Mac mini M6, 16 GB Unified Memory)
Measured directly on Apple Silicon Metal GPU (-ngl 99 --no-host -fa on --cache-type-k q4_0 --cache-type-v q4_0) with DFlash2 speculative decoding and GGML_METAL_PTQ1_MULTICOL=1:
| Configuration | Mode | DFlash2 DAR | Token Latency | Speed | Process RSS | Context |
|---|---|---|---|---|---|---|
| Naive Hybrid v1 (54 layers Q4_1) | Slow | 35.0% | 124.22 ms | 8.05 t/s | 10.85 GiB | 32k |
| Pure Metal LUT (No Speculation) | Baseline | — | 60.56 ms | 16.51 t/s | 7.65 GiB | 32k |
Core16 (n-max 1, p-min 0.58) |
mc_c2 |
87.45% | 52.14 ms | 19.18 t/s | 8.18 GiB | 32k |
Core16 (n-max 2, p-min 0.65) |
mc_c3 |
88.45% | 52.32 ms | 19.11 t/s | 8.18 GiB | 32k |
| Core16 + Full 128k Context | mc_c2 |
93.75% | 47.91 ms | 20.87 t/s | 9.97 GiB | 131,072 |
- Perplexity (PPL):
3.6742 ± 0.3901(vs3.65original FP16 baseline;3.9214on Hybrid v1). - Prompt Prefix Cache (
--cache-ram 1024): Reduces repeated 1k-token prompt eval latency from1005 msto140 ms(-86.1%). - Dynamic Vision Support (
kenzan_router.py): Routes text-only traffic to the leankenzan-qwen3.8:27bbackend and seamlessly switches tokenzan-qwen3.8:27b-vision(--mmprojQ8_0) when images are present in the payload.
Quick Start & Recommended Deployment (Apple Silicon 16 GB)
Important Runtime Requirement: Stock upstream
llama.cppcannot loadPTQ1_0Codebook-LUT weights orDFlash2speculative draft heads. You must use the patchedPrismML-Eng/llama.cppMetal runtime (mac-flint v0.7.0-dev build 12128or newer).
1. Required Binaries & Patched Metal Libraries
Download the model (after signing the gated waiver above), the precompiled macOS ARM64 runtime bundle, and tools/kenzan_router.py:
# After signing the waiver, download with your HF token:
hf download kabcnd/Kenzan-Qwen3.8-27B-Abliterated-GGUF \
Kenzan-Qwen3.8-27B-Abliterated-Core16-Q4_1-M6-16G.gguf \
bin/llama-flint-metal-macos-arm64.tar.gz \
tools/kenzan_router.py --local-dir .
tar -zxvf bin/llama-flint-metal-macos-arm64.tar.gz
export DYLD_LIBRARY_PATH="$(pwd)/llama-flint:$DYLD_LIBRARY_PATH"
2. Mandatory Metal & DFlash2 Environment Variables
Enable mc_c2/mc_c3 multi-column Metal register tiling and fused DFlash2 FFN execution, and clear conflicting pipeline flags:
export GGML_METAL_PTQ1_MULTICOL=1
export LLAMA_DFLASH_FUSED_FFN=1
export LLAMA_SPEC_ARBITER_MODE=active
unset LLAMA_SPEC_PIPELINE LLAMA_SPEC_PIPE_MODE GGML_OP_OFFLOAD_MIN_BATCH
3. Production llama-server Launch Command (128k Context, ~10 GiB RSS)
./llama-flint/llama-server \
-m Kenzan-Qwen3.8-27B-Abliterated-Core16-Q4_1-M6-16G.gguf \
-md Ternary-Bonsai-2-27B-DFlash2-ft5-asym-Q2_K-FC-FullRank.gguf \
--spec-type draft-dflash \
--spec-draft-n-max 1 --spec-draft-n-min 1 --spec-draft-p-min 0.58 --spec-draft-ngl 99 \
--host 127.0.0.1 --port 11435 \
-ngl 99 --no-host -fa on \
-c 131072 --cache-type-k q4_0 --cache-type-v q4_0 -kvu -np 1 \
-t 4 -b 512 -ub 512 --cache-ram 1024 --fit off \
--jinja --reasoning-preserve --metrics \
--temp 0.7 --top-p 0.9 --top-k 40 --min-p 0.05 \
--repeat-penalty 1.02 --presence-penalty 0.15 --frequency-penalty 0.15 \
--alias "kenzan-qwen3.8:27b"
You can also run the unified multi-model proxy tools/kenzan_router.py on port 11434 (kenzan-qwen3.8:27b / kenzan-qwen3.8:27b-vision / kenzan-flint:40b / bonsai-2:latest).
Liability Waiver & Responsible Use Disclaimer
This model is an abliterated (uncensored) research artifact with safety refusal directions removed. By downloading, loading, or executing this model or any other model published by kabcnd / Kenzan Project, you agree that:
- You assume full and exclusive responsibility for all outputs, deployments, and consequences.
- The authors and maintainers (kabcnd / Kenzan Project) are completely released from any and all liability for misuse, damages, or harm arising from this or any other model in the Kenzan / Huihui / Flint series.
- You will use this model solely for lawful, ethical, and authorized research purposes.
Acknowledgements & Inspiration
- Prism ML (https://huggingface.co/prism-ml) — Inspired by and built upon the Bonsai-2 architecture,
PTQ1_0ternary quantization,DFlash2speculative drafting, and the Metal kernel foundations that made 27B/40B inference on 16 GB consumer hardware possible. - Huihui-AI (https://huggingface.co/huihui-ai) — For the upstream
Huihui-Qwen3.8-27B-abliteratedbase model. - Misha — Special thanks to Misha for ongoing support, encouragement, and financial assistance that helped keep this local AI hardware research moving forward.
Support the Project & Funding Goals
I am an independent researcher optimizing frontier 27B–80B models to run locally on affordable consumer hardware without expensive NVIDIA server clusters.
Current Hardware Fundraising Goal: Mac mini 24 GB / 32 GB Unified Memory
- Why it matters: Testing and calibrating 40B–80B MoE models on a base 16 GB Mac mini hits a hard physical ceiling (~12.5 GB wired Metal limit). A Mac mini 24 GB / 32 GB will unlock higher-precision anchor grids, larger KV caches, and full 70B–80B ternary/4-bit hybrid models for the open-source community.
- What your support funds: Direct hardware upgrades, open-source GGUF releases, Metal kernel benchmarks, and reproducible quantization tooling.
☕ Support Local AI Research on Ko-fi →
Even a small contribution or sharing this repository helps enormously. Thank you! ❤️