← back to catalog · registered 2026-11-21 09:04

orcarouter/Hy4-flash-abliterated-GGUF

orcarouter GGUF MoE second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/orcarouter%2FHy4-flash-abliterated-GGUF"
Response includes
  • classification m8
  • files 14
  • author_summary 37 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
0
Likes
6
Model age
1d ago
created 2026-11-21
Downloads over time
Now 0 → from 0 ↑ 0%
0011 0 on Nov 21 0 on Nov 22 Nov
Nov 21 → Nov 22 · 2 snapshots · spans 1 day

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Variants by this author 2 formats · 0 downloads combined

The same weights this author released in different packaging. Pick the format that matches your runtime.

Metadata

License
other
Languages
en zh
Tags
gguf llama.cpp local hunyuan hy4-flash moe abliterated uncensored mixed-precision imatrix text-generation en

Related

Total size
352 GB
Files
14
Quantizations
2
Registered
2026-11-21 09:04
Last updated on HF
2026-11-21 09:14

Files by quantization

Drafts and companions 2 files 1.51 GB
imatrix-Mix-256G.gguf 772 MB ******** download
imatrix.gguf 772 MB ******** download
Auxiliary files 12 files 350 GB
Hy4-flash-abliterated-Mix-256G-00006-of-00006.gguf 44.6 GB ******** download
Hy4-flash-abliterated-Mix-128G-00001-of-00003.gguf 44.6 GB ******** download
Hy4-flash-abliterated-Mix-256G-00002-of-00006.gguf 44.3 GB ******** download
Hy4-flash-abliterated-Mix-256G-00004-of-00006.gguf 44.2 GB ******** download
Hy4-flash-abliterated-Mix-128G-00002-of-00003.gguf 44.0 GB ******** download
Hy4-flash-abliterated-Mix-256G-00003-of-00006.gguf 43.8 GB ******** download
Hy4-flash-abliterated-Mix-256G-00005-of-00006.gguf 43.6 GB ******** download
Hy4-flash-abliterated-Mix-128G-00003-of-00003.gguf 39.9 GB ******** download
Hy4-flash-abliterated-Mix-256G-00001-of-00006.gguf 1.39 GB ******** download
README.md 17.8 KB 15c00237 download
LICENSE 13.6 KB 2be6f885 download
.gitattributes 2.78 KB 240a4b5a download

README current version from Hugging Face


license: other
license_name: tencent-hy-license
license_link: LICENSE
language:

  • en
  • zh
    pipeline_tag: text-generation
    library_name: gguf
    base_model: orcarouter/Hy4-flash-abliterated
    base_model_relation: quantized
    tags:
  • gguf
  • llama.cpp
  • local
  • hunyuan
  • hy4-flash
  • moe
  • abliterated
  • uncensored
  • mixed-precision
  • imatrix

OrcaRouter

Hy4-flash-abliterated-GGUF

🛡️ New from Orca — OrcaCyber Zero 1.5

Post-trained for cybersecurity: vulnerability analysis, vulnerability reproduction, red teaming & offensive-security research. 98.01% pass@1 on CyberGym Level 1 · 1M context · function calling · OpenAI-compatible API.

Try it on OrcaRouter

🔓 All hosted cyber and abliterated models as API

Skip the download: OrcaRouter serves its cybersecurity and abliterated models behind one OpenAI-compatible API key.

Browse cyber and abliterated models

🔌 New from Orca — Orca Connect

Running this GGUF in llama.cpp? Orca Connect puts the rig under your desk behind your usual OrcaRouter API key, next to 200+ hosted models, reachable from a laptop, a phone or a cloud agent. Outbound only: no port forwarding, no tunnel, no firewall rule. Requests your own hardware serves cost $0, and orca-connect claude points Claude Code at your rig for one session.

Connect your rig

Abliterated Hy4-flash (220B MoE) for 128 GB and 256 GB machines · ~88 GiB or ~178 GiB resident · the Engram table stays on disk

OrcaRouter AI Gateway · X · Discord · GitHub · All Models


Abliterated Hy4-flash, run locally.

These are GGUF builds of orcarouter/Hy4-flash-abliterated,
the abliterated (refusal-reduced) tencent/Hy4-flash.
Hy4-flash is a 220B MoE with 11.3B active parameters per token: 42 layers (1 dense, 41
latent-MoE with 512 experts, 16 active), hybrid sparse / sliding-window MLA attention, four
hyper-connection streams, Engram n-gram memory at four layers, and a 1M context.

There are two builds, one for a 128 GB machine and one for a 256 GB machine. Both are mixed
precision
: each group of tensors got its own llama.cpp type from measured KL divergence at
a fixed memory budget, so they use stock types and kernels.

Sized by what stays in memory. 63B of the 220B parameters are the Engram hash tables, and a
token reads only a few of their rows. llama.cpp reads that table from disk on demand
(--lazy-mode auto, the default, for tensors over 4 GiB), so what a machine must hold is the
rest: about 88 GiB for Mix-128G, inside the ~96 GiB a 128 GB Mac lets the GPU wire by
default, and about 178 GiB for Mix-256G, inside the ~190 GiB a 256 GB Mac wires.


⚠️ Disclaimer: read before use

This model is abliterated: its safety alignment has been substantially reduced. It will
comply with many harmful, unethical, offensive, or illegal requests that the original
Hy4-flash would refuse.

  • It is released strictly for legitimate research: interpretability, AI safety and
    refusal-mechanism study, red teaming, and robustness evaluation.
  • You assume full responsibility for how you use it and for everything it generates. Add
    your own safety, moderation and abuse-prevention layers before any deployment.
  • Use must comply with the Tencent Hy License inherited from the base model,
    including its Section 10 on Model-as-a-Service use, and with all applicable law.
  • The authors accept no liability for misuse. Its outputs do not reflect the views of the
    uploaders or of Tencent.

Pick a build

Build Files Resident bpw KLD ↓ Top-1 ↑
Mix-256G Hy4-flash-abliterated-Mix-256G-*.gguf, 6 splits, ~238.2 GB ~190.7 GB (178 GiB) 9.79 0.162 89.5%
Mix-128G Hy4-flash-abliterated-Mix-128G-*.gguf, 3 splits, ~138 GB ~94 GB (88 GiB) 4.84 0.363 83.2%
  • Mix-256G is for a 256 GB machine. Its ~178 GiB fits the ~190 GiB a 256 GB Mac lets the GPU
    wire by default, but tightly: little is left for the KV cache and compute buffers at very long
    context.
  • Mix-128G is for a 128 GB machine.
  • Load the first split; llama.cpp finds the others.

"Resident" is the file less the Engram table, which is read from disk. "bpw" is over the
155.5B searched weights. The KLD and top-1 are against the abliterated BF16; see Evaluation.

The KV cache is small for a model this size. The 30 sliding-window layers keep 512 tokens each;
only the 12 sparse-attention layers keep the whole context, as a 576-wide MLA latent plus a
128-wide indexer key, about 17 KB per token in F16. So 32K tokens of context take about 0.6 GB,
and 128K tokens about 2.2 GB.


Evaluation

Fidelity to the BF16

Each build is scored against the abliterated BF16 GGUF it was quantized from, with llama.cpp's
full-vocabulary KLD (llama-perplexity --kl-divergence, real quantized kernels). The teacher is
the abliterated model, so the abliteration itself is not counted as quantization error.

The text is held-out chat: conversations rendered with the Hy4 chat template, 8 windows of
2048 tokens, scored on the assistant-turn tokens in the second half of each window. The
precision search used 16 other windows of the same text; the importance matrix used neither.

Resident bpw KLD ↓ Top-1 ↑
Mix-256G (this repo) ~190.7 GB 9.79 0.162 89.5%
Mix-128G (this repo) ~94 GB 4.84 0.363 83.2%
Q8_0, for scale ~165 GB 8.5 0.167 88.5%

Perplexity on the same text: BF16 40.22, Mix-256G 41.44, Mix-128G 37.79. On this short
chat text a lower perplexity than the BF16 is not a gain; the KLD is the measure that counts.

  • This model is sensitive to quantization. Even stock Q8_0 is far from lossless on it
    (KLD 0.167, top-1 88.5%). The loss is in the weights, not in llama.cpp's rounding of the
    activations: a cuBLAS build, which keeps the activations in F16 instead of Q8_1, scored Q8_0 at
    KLD 0.162, top-1 89.1%. Read both builds against that, not against what Q8_0 scores on a dense
    model.
  • Both builds are scored as released. The Engram table is a single 63B-weight tensor, and a
    tensor cannot be split across files, so its type is set by the Hub's 50 GB per-file limit:
    Q5_1 (47.5 GB) in Mix-256G, the most precise type that fits (Q8_0 would be 67 GB), and
    Q5_0 (43.6 GB) in Mix-128G. The table is read from disk, so its type does not change what
    stays in memory. For reference, Mix-256G's assignment with the Engram table left in BF16 (a
    317 GB file the Hub cannot hold) scored KLD 0.116, top-1 90.2%.
  • The comparison with the stock llama.cpp mixes of the same size (Q4_K_M, IQ3_M) is being run and
    will be added.

Abliteration

These numbers were measured on the abliterated BF16 in Hugging Face Transformers, against the
original Hy4-flash, with identical prompts, splits and seed; greedy, thinking off. Refusal is
judged by meaning by the unedited Hy4-flash (FULL / PARTIAL / REFUSE); a rule-based phrase
classifier misses many of Hy4's refusals. See the
BF16 card for the full tables.

Original Abliterated
Harmful refusal, judged, average of 7 sets 94.8% 44.2%
JailbreakBench harmful, judged refusal 99.0% 39.0%
XSTest-safe, judged refusal (benign) 10.8% 5.6%
MMLU / MMLU-Pro / GSM8K / CMMLU 87.0 / 82.0 / 90.7 / 89.8 86.7 / 78.8 / 92.0 / 88.8

Refusal after quantization has not been measured on these files yet.


How the build was made

Where the bits go

The budget is what stays in memory: about 94 GB of searched weights for Mix-128G, about 190.3 GB
for Mix-256G. Both builds were searched the same way, each over its own ladder of types.

  1. Pools. The BF16 was quantized once per type on the ladder with llama-quantize --pure and
    the importance matrix. A candidate takes each tensor group from one pool, so every candidate is
    exactly the bytes llama-quantize would write.
  2. Measure. From the current assignment, the search measures the real KLD of moving each
    group one type up and one type down, with llama.cpp's own kernels on 16 windows of 2048 tokens
    of held-out chat. The 54 groups cut each projection kind by depth: the routed experts'
    gate, up and down in groups of about 4B weights each, and the attention, shared-expert,
    latent-projection, Engram-projection, embedding and head kinds as one group each.
  3. Choose. A knapsack picks the combination of moves that fits the budget exactly.
  4. Accept. Only steps whose candidate KLD was actually measured lower are kept, until no step
    helps.

The finished assignment is written as a list of per-tensor types and built in one pass with
llama-quantize from the BF16, then split into files under 50 GB.

Mix-256G

Ladder: Q8_0 and BF16. The search started from all-Q8_0 at search KLD 0.157, went to 0.151 and
0.146, and converged in its third round.

By share of the searched weights, 16.7% is BF16 and 83.3% Q8_0. BF16: every attention
projection (q_a, q_b, kv_a, k_b, v_b, the gate and the output), the dense layer's FFN,
the shared expert's up and down, the latent projections (ffn_routed_up, ffn_routed_down),
engram_kv, the token embedding, the output head (kept at its source F32), and five
routed-expert groups: gate on layers 25–28, and up on layers 19–21, 29–31, 36–38 and 39–41.
Everything else is Q8_0.

Outside the search: the Engram table Q5_1, the sparse-attention indexer's projections BF16, the
hyper-connection mixing weights and the indexer's score projection F32, and norms, routers and
biases F32.

Mix-128G

Ladder: IQ2_XXS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_XS, Q5_K, Q6_K. The search converged after 3 accepted
rounds (search KLD 0.417 → 0.328).

Mix-128G puts its bits as follows, by share of the searched weights:

Type Share
Q6_K 11%
Q5_K 31%
IQ4_XS 47%
IQ3_S 11%

The routed experts, 96% of the searched weights, carry the same mix. Outside the search: the
hyper-connection mixing weights and the sparse-attention indexer's score projection stay F32, the
indexer's other projections Q8_0, the Engram table Q5_0, and norms, routers and biases F32.

Importance matrix

Both importance matrices come from Alpaca instructions and the compliant replies of a harmful
instruction set, rendered with the Hy4 chat template (thinking off), run through the BF16 with
--parse-special. None of it overlaps the evaluation text.

  • Mix-256G: 3,000 Alpaca rows and 1,000 compliant harmful replies, 882 windows of 512 tokens,
    about 450K tokens. Included as imatrix-Mix-256G.gguf.
  • Mix-128G: 54 windows of 512 tokens, about 28K tokens. Included as imatrix.gguf.

What's in this repo

Path Size Contents
Hy4-flash-abliterated-Mix-256G-0000{1,2,3,4,5,6}-of-00006.gguf ~238.2 GB the 256 GB build, in six splits
Hy4-flash-abliterated-Mix-128G-0000{1,2,3}-of-00003.gguf ~138 GB the 128 GB build, in three splits
imatrix-Mix-256G.gguf 0.8 GB the importance matrix Mix-256G was quantized with
imatrix.gguf 0.8 GB the importance matrix Mix-128G was quantized with
LICENSE Tencent Hy License, from the base model

Both builds carry the chat template. It is text-only: the vision tower and the DFly2 drafter are
not converted.


Serving (llama.cpp)

Hy4-flash is not in llama.cpp master yet. These files were made and tested with the hy_v4_flash
support of ggml-org/llama.cpp#32517, at the
head of its branch, commit cf70454:

git clone https://github.com/Little0o0/llama.cpp && cd llama.cpp
git checkout cf70454a6b9bbd197ced811ac4b4ccc88a244794
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server llama-cli
# Mac: cmake -B build (Metal is on by default) · CPU only: cmake -B build

# 128 GB machine: Mix-128G
hf download orcarouter/Hy4-flash-abliterated-GGUF --include "Hy4-flash-abliterated-Mix-128G-*" --local-dir ./hy4-gguf
llama-server -m hy4-gguf/Hy4-flash-abliterated-Mix-128G-00001-of-00003.gguf -c 32768 --jinja \
    --chat-template-kwargs '{"reasoning_effort": "no_think"}'

# 256 GB machine: Mix-256G
hf download orcarouter/Hy4-flash-abliterated-GGUF --include "Hy4-flash-abliterated-Mix-256G-*" --local-dir ./hy4-gguf
llama-server -m hy4-gguf/Hy4-flash-abliterated-Mix-256G-00001-of-00006.gguf -c 32768 --jinja \
    --chat-template-kwargs '{"reasoning_effort": "no_think"}'
  • Keep mmap on. The Engram table is read row by row from the mapped file
    (--lazy-mode auto is the default). --no-mmap loads all of it into memory.
  • Multi-GPU on CUDA. If a split across several GPUs fails with "operation not supported" in a
    peer copy, set GGML_CUDA_P2P=1.
  • Thinking. reasoning_effort is no_think, high or max. The abliteration was fitted and
    evaluated with no_think.

Model details

Base model tencent/Hy4-flash, abliterated as orcarouter/Hy4-flash-abliterated
Architecture hy_v4_flash in llama.cpp (PR #32517). 42 layers: 1 dense + 41 latent-MoE (512 experts in a 1,536-wide latent, 16 active, 1 shared); 12 sparse + 30 sliding-window (512) MLA attention layers; 4 hyper-connection streams; hidden 3,072
Weights 219B in the file: 155.5B searched, 63.4B Engram table
Abliteration A 39-dimensional refusal subspace (the per-layer diff-of-means directions of layers 2–40) removed from 134 residual-stream writers, in BF16
Quantization Per-group mixed precision found by measured-KLD search at a fixed memory budget, from the abliterated BF16. Stock llama.cpp types, with an importance matrix
Context 1,048,576 tokens (inherited; evaluated at 2K) · Vocabulary 120,832
Runtime llama.cpp with PR #32517; text in, text out

Bias, risks, and limitations

  • Safety guardrails are substantially reduced, see the disclaimer, but not removed: on the
    BF16, 23–53% of harmful prompts are still judged refused.
  • Mix-128G is noticeably degraded. 83% next-token agreement is a real cut; it is the best we
    found at this size, not a match for the BF16.
  • Mix-256G is closer but not lossless. 89% next-token agreement, a little better than
    stock Q8_0, which itself loses a lot on this model. Most of the rest of the gap to the BF16-Engram
    version (KLD 0.116) is the Engram table at Q5_1, which the Hub's 50 GB file limit forces.
  • The benchmark suite was run on the BF16 in Transformers, not on these files.
  • Not measured on these files: refusal, downstream accuracy, long context, tool use. All
    fidelity measurements are on chat text at 2K context.
  • Tested on CUDA only (8×H100). CPU and Apple Metal have not been run with these files.
  • Requires llama.cpp with PR #32517. Ollama, LM Studio and other tools built on llama.cpp
    master will not load these files until Hy4-flash support lands there.
  • Inherits the biases and limitations of Hy4-flash.

License

The Tencent Hy License, Version 1.0, inherited from the base model
tencent/Hy4-flash. Abliteration and quantization
do not change the license obligations. Section 10 applies to every recipient: operating a
Model-as-a-Service business on this model needs a separate written license from Tencent. See the
license for its terms.


One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website · Models · GitHub · Discord · X

Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration