license: apache-2.0
base_model: orcarouter/Flint-abliterated-GGUF
tags:
- kenzan
- gguf
- ternary
- moe
- 40b
- abliterated
- uncensored
- apple-silicon
- metal
- hadamard
- ptq1_0
- q4_0
- speculative-decoding
- kenzan-router
pipeline_tag: text-generation
language: - en
- ru
- zh
inference: false
extra_gated_heading: "Mandatory Liability Waiver & Uncensored Model Agreement"
extra_gated_prompt: >
WARNING: This repository hosts an ABLITERATED (uncensored) 40B MoE language model with safety refusals removed.
By requesting access to, downloading, or using this model (or any other models published by kabcnd / the Kenzan Project),
you explicitly acknowledge, sign, and agree to the following legally binding terms:
(1) SOLE RESPONSIBILITY: You assume 100% personal and legal responsibility for all prompts, outputs, deployments, and consequences arising from the use of this model or any other models provided by kabcnd / the Kenzan Project.
(2) FULL RELEASE OF LIABILITY: The author(s), maintainer(s), and contributors (kabcnd / Kenzan Project) bear ZERO liability and are hereby completely released and held harmless from any claims, damages, misuse, illegal acts, or direct/indirect harm caused by this model or any other Kenzan / Huihui / Flint models.
(3) LAWFUL USE ONLY: You will use this model strictly for lawful research, red-teaming, security evaluation, or authorized local testing in compliance with all applicable laws in your jurisdiction.
extra_gated_fields:
Full Name or Signature (required to sign waiver): text
Organization / Affiliation (or Independent Researcher): text
I confirm that I am at least 18 years old: checkbox
I accept 100% personal responsibility and release kabcnd / Kenzan Project from all liability for any harm caused by this or other models: checkbox
extra_gated_button_content: "Sign Liability Waiver & Unlock Download"
☕ Support Local AI Development: Help me acquire a Mac mini 24GB/32GB to optimize and test local MoE models up to 80B parameters. Donate via Ko-fi →
[!CAUTION]
Kenzan-Flint-40B-Abliterated-Hepta48Up — "40B Uncensored MoE, 9.77 GiB, 79 t/s, Zero Guardrails."
We took an 11.17 GiB abliterated 40B MoE, rotated 141 expert tensors throughFWHT_512Hadamard surgery, pinned theHepta48Uptail grid inQ4_0, fixed the hiddenexpert_weights_scaletrap, and squeezed the whole thing into 9.77 GiB so a base 16 GB Mac mini can sprint at nearly 80 tokens/sec without a single refusal.
Kenzan-Flint-40B-Abliterated-Hepta48Up (9.77 GiB GGUF)
Kenzan-Flint-40B-Abliterated-Hepta48Up (Kenzan-Flint-40B-Abliterated-Hepta48Up-Q4_0-M6-16G.gguf) is a hybrid ternary-4bit Mixture-of-Experts (MoE) uncensored language model (40B total parameters, 256 routed experts + 2 shared experts, 49 layers) distilled and compressed from orcarouter/Flint-abliterated-GGUF (Flint-abliterated-Mix-12G.gguf, 11.17 GiB) specifically for Apple Silicon 16 GB Unified Memory (Mac mini M6).
Short Summary:
Kenzan-Flint-40B-Abliterated-Hepta48Up (9.77 GiB GGUF) — Hybrid PTQ1_0 (1.75 bpw) + Q4_0+FWHT_512 40B uncensored MoE tuned for Apple Silicon 16 GB Unified Memory, achieving 29.38 WikiText-2 holdout PPL, 69.7–79.4 t/s generation, and 262k context at 11.82 GiB RSS.
Architecture & Pin-Grid Surgery
- Ternary Crown (
PTQ1_0, 1.75 bpw): All 133 non-anchor routed expert tensors (blk.2..blk.48) are dequantized fromFlint-abliterated-Mix-12G.gguf, rotated via block-512 Fast Walsh-Hadamard Transform (FWHT_512), quantized withflint_imatrix.datforce-density (0.691active trits), and stabilized with permanent Salient Clipping (κ = 0.15, η = -0.025). - Pin Grid (
Hepta48Up):down_expsanchored at layers{8, 12, 20, 40, 44, 46, 48}+ output-adjacentblk.48.ffn_up_expsinQ4_0+FWHT_512. - Scale Alignment (
flint.expert_weights_scale = 1.0): Aligned GGUF expert scaling to match the abliterated donor (1.0instead of1.24). Always-On Backbone: High-precision imatrix-calibratedQ8_0token_embd,Q6_Koutput.weight,Q5_K/IQ4_XS/IQ3_XXSattention and shared experts.
Verified Benchmarks (Mac mini M6, 16 GB Unified Memory)
Measured directly on Apple Silicon Metal GPU (-ngl 99 --no-host -fa on -c 262144 --cache-type-k q4_0 --cache-type-v q4_0) with ngram-map-k (3x4) speculative decoding:
| Metric | Flint-abliterated-Mix-12G (Donor) |
Kenzan-Flint-40B-Abliterated-Hepta48Up |
|---|---|---|
| GGUF File Size | 11.168 GiB |
9.768 GiB (-1.40 GiB) |
Process RSS (-c 262144) |
N/A | 11.82 GiB (12,392,128 KB) |
WikiText-2 Holdout (c512) PPL |
28.1418 ± 0.9639 |
29.3767 ± 0.8194 |
Generation Throughput (tg) |
— | 69.7 – 79.4 t/s (live router 11434) |
Prompt Ingestion (pp) |
— | 190.1 – 275.2 t/s |
Quick Start & Recommended Deployment (Apple Silicon 16 GB)
Important Runtime Requirement: Stock upstream
llama.cppcannot loadKenzanGGUF files because they use packed ternaryPTQ1_0weights andprism.hadamard.*phase-rotation metadata. You must use the patchedPrismML-Eng/llama.cppruntime (mac-flint v0.7.0-dev build 12128or newer).
1. Required Binaries & Patched Metal Libraries
Download the model (after signing the gated waiver above), the precompiled macOS ARM64 runtime bundle, and tools/kenzan_router.py:
hf download kabcnd/Kenzan-Flint-40B-Abliterated-Hepta48Up-GGUF \
Kenzan-Flint-40B-Abliterated-Hepta48Up-Q4_0-M6-16G.gguf \
bin/llama-flint-metal-macos-arm64.tar.gz \
tools/kenzan_router.py --local-dir .
tar -zxvf bin/llama-flint-metal-macos-arm64.tar.gz
export DYLD_LIBRARY_PATH="$(pwd)/llama-flint:$DYLD_LIBRARY_PATH"
2. Mandatory Metal Environment Variables
export GGML_METAL_PTQ1_MULTICOL=1
export GGML_METAL_FUSE_GATHER_MV=1
export GGML_METAL_TENSOR_ENABLE=1
export GGML_METAL_SHARED_BUFFERS_ENABLE=1
export GGML_METAL_Q1_0_POPCNT=1
3. Production llama-server Launch Command (16 GB Safe, 262k Context)
./llama-flint/llama-server \
-m Kenzan-Flint-40B-Abliterated-Hepta48Up-Q4_0-M6-16G.gguf \
--host 127.0.0.1 --port 11435 \
-ngl 99 --no-host -fa on \
-c 262144 --cache-type-k q4_0 --cache-type-v q4_0 -np 1 -kvu \
-t 4 --threads-draft 2 -b 512 -ub 512 --ctx-checkpoints 8 --cache-ram 2048 \
--jinja --metrics \
--spec-type ngram-map-k --spec-ngram-map-k-size-n 3 --spec-ngram-map-k-size-m 4 \
--spec-draft-n-max 3 --spec-draft-n-min 1 \
--samplers "top_k;dry;penalties;top_p;min_p;temperature" \
--temp 0.20 --top-p 0.95 --top-k 40 --min-p 0.08 \
--repeat-penalty 1.05 --repeat-last-n 256 \
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 3 --dry-penalty-last-n 512 --dry-sequence-breaker $'\f' \
--reasoning on --reasoning-format deepseek --reasoning-budget -1 --reasoning-preserve \
--alias "kenzan-flint:40b-abliterated"
You can also run the unified multi-model proxy tools/kenzan_router.py on port 11434 (kenzan-flint:40b-abliterated, kenzan-flint:40b, kenzan-qwen3.8:27b, bonsai-2:latest).
Liability Waiver & Responsible Use Disclaimer
This model is an abliterated (uncensored) research artifact with safety refusal directions removed. By downloading, loading, or executing this model or any other model published by kabcnd / Kenzan Project, you agree that:
- You assume full and exclusive responsibility for all outputs, deployments, and consequences.
- The authors and maintainers (kabcnd / Kenzan Project) are completely released from any and all liability for misuse, damages, or harm arising from this or any other model in the Kenzan / Huihui / Flint series.
- You will use this model solely for lawful, ethical, and authorized research purposes.
Acknowledgements & Inspiration
- Prism ML (https://huggingface.co/prism-ml) — Inspired by and built upon the Bonsai-2 architecture,
PTQ1_0ternary quantization,FWHT_512Hadamard transforms, and Metal kernel foundations. - OrcaRouter (https://huggingface.co/orcarouter/Flint-abliterated-GGUF) — For the upstream
Flint-abliterated-Mix-12Gdonor model. - Misha — Special thanks to Misha for ongoing support, encouragement, and financial assistance that helped keep this local AI hardware research moving forward.
Support the Project & Funding Goals
I am an independent researcher optimizing frontier 27B–80B models to run locally on affordable consumer hardware without expensive NVIDIA server clusters.
Current Hardware Fundraising Goal: Mac mini 24 GB / 32 GB Unified Memory
- Why it matters: Testing and calibrating 40B–80B MoE models on a base 16 GB Mac mini hits a hard physical ceiling (~12.5 GB wired Metal limit). A Mac mini 24 GB / 32 GB will unlock higher-precision anchor grids, larger KV caches, and full 70B–80B ternary/4-bit hybrid models for the open-source community.
- What your support funds: Direct hardware upgrades, open-source GGUF releases, Metal kernel benchmarks, and reproducible quantization tooling.
☕ Support Local AI Research on Ko-fi →
Even a small contribution or sharing this repository helps enormously. Thank you! ❤️