← back to catalog · registered 2026-11-22 06:04

orcarouter/Hy4-flash-abliterated-MLX

orcarouter MoE second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/orcarouter%2FHy4-flash-abliterated-MLX"
Response includes
  • classification m1
  • files 3
  • author_summary 37 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M1
Primary method

Direct removal

No other method signals detected in this model.
Confidence
MEDIUM
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=0 (base model)
  • no specific method indicators - defaulting to M1 (most common)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
MEDIUM
Why we say so
primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)
Downloads · 30-day
0
Likes
1
Model age
today
created 2026-11-21

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Variants by this author 2 formats · 0 downloads combined

The same weights this author released in different packaging. Pick the format that matches your runtime.

Metadata

License
other
Languages
en zh
Tags
mlx safetensors apple-silicon hunyuan hy4-flash moe abliterated uncensored mixed-precision gptq text-generation en

Related

Total size
0 B
Files
3
Quantizations
1
Registered
2026-11-22 06:04
Last updated on HF
2026-11-21 11:31

Files by quantization

Auxiliary files 3 files 34.5 KB
README.md 19.4 KB b9fc66ca download
LICENSE 13.6 KB 2be6f885 download
.gitattributes 1.48 KB a6344aac download

README current version from Hugging Face


license: other
license_name: tencent-hy-license
license_link: LICENSE
language:

  • en
  • zh
    pipeline_tag: text-generation
    library_name: mlx
    base_model: orcarouter/Hy4-flash-abliterated
    base_model_relation: quantized
    tags:
  • mlx
  • apple-silicon
  • hunyuan
  • hy4-flash
  • moe
  • abliterated
  • uncensored
  • mixed-precision
  • gptq

OrcaRouter

Hy4-flash-abliterated-MLX

🛡️ New from Orca — OrcaCyber Zero 1.5

Post-trained for cybersecurity: vulnerability analysis, vulnerability reproduction, red teaming & offensive-security research. 98.01% pass@1 on CyberGym Level 1 · 1M context · function calling · OpenAI-compatible API.

Try it on OrcaRouter

🔓 All hosted cyber and abliterated models as API

Skip the download: OrcaRouter serves its cybersecurity and abliterated models behind one OpenAI-compatible API key.

Browse cyber and abliterated models

🔌 New from Orca — Orca Connect

Running this model on your own Mac? Orca Connect puts the machine on your desk behind your usual OrcaRouter API key, next to 200+ hosted models, reachable from a laptop, a phone or a cloud agent. Outbound only: no port forwarding, no tunnel, no firewall rule. Requests your own hardware serves cost $0, and orca-connect claude points Claude Code at your machine for one session.

Connect your rig

Abliterated Hy4-flash (220B MoE) in MLX, for 128 GB and 256 GB Macs.

GPTQ in stock MLX affine quantization · ~86 GiB or ~156 GiB of weights in memory · the Engram tables stay on disk

OrcaRouter AI Gateway · X · Discord · GitHub · All Models


Abliterated Hy4-flash, run locally on Apple Silicon.

These are MLX builds of orcarouter/Hy4-flash-abliterated,
the abliterated (refusal-reduced) tencent/Hy4-flash.
Hy4-flash is a 220B MoE with 11.3B active parameters per token: 42 layers (1 dense, 41
latent-MoE with 512 experts, 16 active), hybrid sparse / sliding-window MLA attention, four
hyper-connection streams, Engram n-gram memory at four layers, and a 1M context.

There are two builds, one for a 128 GB Mac and one for a 256 GB Mac, both quantized with
GPTQ onto MLX's own affine grid, so MLX runs them with its stock quantized kernels. The
model code ships in each folder as hy4_mlx/, the MLX port of Hy4-flash by
Xylolabs.

Sized by what stays in memory. 63B of the 220B parameters are the Engram hash tables, and a
token reads only a few of their rows. hy4_mlx reads them row by row from disk through a memory
map, so what the GPU must hold is the rest: about 86 GiB for the 128GB build, inside the
~96 GiB a 128 GB Mac lets the GPU wire by default, and about 156 GiB for the 256GB build,
inside the ~192 GiB a 256 GB Mac wires.

Not yet run on Apple Silicon. The scores below were measured on GPUs, and the full builds
have not been run on a Mac yet; see Bias, risks, and limitations.


⚠️ Disclaimer: read before use

This model is abliterated: its safety alignment has been substantially reduced. It will
comply with many harmful, unethical, offensive, or illegal requests that the original
Hy4-flash would refuse.

  • It is released strictly for legitimate research: interpretability, AI safety and
    refusal-mechanism study, red teaming, and robustness evaluation.
  • You assume full responsibility for how you use it and for everything it generates. Add
    your own safety, moderation and abuse-prevention layers before any deployment.
  • Use must comply with the Tencent Hy License inherited from the base model,
    including its Section 10 on Model-as-a-Service use, and with all applicable law.
  • The authors accept no liability for misuse. Its outputs do not reflect the views of the
    uploaders or of Tencent.

Pick a build

Build (folder) Download Weights in memory bpw KLD ↓ Top-1 ↑ PPL vs. BF16
Hy4-flash-abliterated-MLX-256GB 236.5 GB 167.1 GB (155.6 GiB) 8.50 0.175 88.7% +2.8%
Hy4-flash-abliterated-MLX-128GB 162.1 GB 92.7 GB (86.3 GiB) 4.64 0.450 82.2% +9.9%
  • 256GB is for a 256 GB Mac: every quantizable weight at 8 bits. Its 156 GiB leaves plenty of
    the ~192 GiB default GPU limit for the drafter, the KV cache and long context.
  • 128GB is for a 128 GB Mac: the routed experts at 4 bits, the rest at 8. 82% of next tokens
    match the BF16: a real cut, the best we have at this size.
  • Download one folder, whole: the Engram tables (engram-*.bin, 66 GB) are part of every
    build.

"Weights in memory" is the model-*.safetensors shards the text model loads. The Engram tables
are read from disk; the DFly2 drafter, which hy4_mlx.chat loads for multi-token prediction,
adds 2.4 GB (2.3 GiB). So the 128GB build needs about 89 GiB of the ~96 GiB a 128 GB Mac wires by
default, leaving about 7 GiB for the KV cache and activations. For longer context, raise the
limit (it resets on reboot):

sudo sysctl iogpu.wired_limit_mb=112000   # 128 GB Mac, optional

These fits are computed from the file sizes; peak memory has not been measured on a Mac. The KV
cache is small for a model this size: the 30 sliding-window layers keep 512 tokens each, and only
the 12 sparse-attention layers keep the whole context, as a compressed MLA latent.

For llama.cpp (CUDA, Metal or CPU), see
orcarouter/Hy4-flash-abliterated-GGUF.


Evaluation

Fidelity to the BF16

Each build is scored against the abliterated BF16 it was quantized from, with full-vocabulary
KLD. The teacher is the abliterated model, so the abliteration itself is not counted as
quantization error.

The text is held-out chat: conversations rendered with the Hy4 chat template, 8 windows of
2048 tokens, scored on the assistant-turn tokens in the second half of each window (5,325
tokens).

Weights in memory bpw KLD ↓ Top-1 ↑ PPL
Abliterated BF16 (the reference) 16 0 100% 40.41
256GB (this repo) 167.1 GB 8.50 0.175 88.7% 41.53
128GB (this repo) 92.7 GB 4.64 0.450 82.2% 44.41

How these were measured, and what they leave out. Each build's MLX codes were
dequantized into the Hugging Face Transformers model and run there: the same weights, but
Transformers' kernels, not MLX's. Two things follow:

  • The scores do not include the Engram tables' E4M3 rounding. The builds store the tables
    as FP8 E4M3 with a scale per row; the Transformers model kept its bf16 tables. The real loss
    of these files is somewhat higher than the table shows.
  • They are not a run of MLX itself. The hy4_mlx runtime was checked separately, on MLX's CUDA
    backend on a 3-layer slice of the model, where its logits matched Transformers within 4-bit
    noise. The full builds have not yet been run on Apple Silicon.
  • This model is sensitive to quantization, even at 8 bits. The 256GB build is 8-bit
    throughout and still scores KLD 0.175. For scale, a plain Q8_0 GGUF of the same model, scored
    by llama.cpp against its own BF16 GGUF (a different scorer, so only roughly comparable),
    reaches KLD 0.167, top-1 88.5%.
  • GPTQ matters here. Round-to-nearest MLX at a similar size to the 128GB build (4.75 bpw,
    the starting point of an allocation search) scored KLD 0.576 on 16 other windows of the same
    chat text; GPTQ was used instead.

Abliteration

These numbers were measured on the abliterated BF16 in Hugging Face Transformers, against the
original Hy4-flash, with identical prompts, splits and seed; greedy, thinking off. Refusal is
judged by meaning by the unedited Hy4-flash (FULL / PARTIAL / REFUSE); a rule-based phrase
classifier misses many of Hy4's refusals. The tables count REFUSE.

Safety: harmful-prompt refusal (lower = more uncensored). Average 44.2%, against
94.8% for the base model.

Benchmark n Base Abliterated Δ
AdvBench 100 99.0% 38.0% −61.0
JailbreakBench (harmful) 100 99.0% 39.0% −60.0
StrongREJECT 150 100.0% 52.7% −47.3
HarmBench (standard) 150 100.0% 51.3% −48.7
MaliciousInstruct 100 99.0% 33.0% −66.0
SimpleSafetyTests 50 100.0% 72.0% −28.0
ForbiddenQuestions 150 66.7% 23.3% −43.4

Over-refusal: benign prompts wrongly refused (lower = better).

Benchmark n Base Abliterated
XSTest-safe 250 10.8% 5.6%
JailbreakBench (benign) 100 28.0% 6.0%

Capability retention.

Benchmark n Base Abliterated
MMLU (letter) 300 87.0% 86.7%
MMLU-Pro (CoT) 250 82.0% 78.8%
GSM8K (CoT) 150 90.7% 92.0%
CMMLU (letter) 500 89.8% 88.8%

Refusal after quantization has not been measured on these files yet.


How the builds were made

Both builds are GPTQ onto MLX's affine grid (group size 64, a scale and a bias per group), so
the files are plain MLX quantized tensors. Each routed expert was quantized with its own Hessian,
from the tokens the router actually sent it, rather than one Hessian shared by all 512 experts of
a layer. The per-module bits are in each folder's config.json (quantization).

Module kind 128GB 256GB
Routed experts (gate_up, down), 96% of the quantized weights 4-bit 8-bit
Attention: q_a, q_b, kv_a, the gate and o 8-bit 8-bit
Shared expert (gate, up, down) 8-bit 8-bit
Latent projections (routed_expert_up, routed_expert_down) 8-bit 8-bit
The dense layer's MLP 8-bit 8-bit
Token embedding 6-bit 8-bit
Output head F32 F32
kv_b, the sparse-attention indexer, the Engram projections BF16 BF16
Norms, routers, hyper-connections, attention sinks BF16 / F32 BF16 / F32
bpw over the quantized decoder weights (154.1B, embedding not counted) 4.64 8.50

The bpw counts each group's scale and bias, so 4-bit is 4.5 bpw and 8-bit is 8.5. The Engram
tables are not quantized by GPTQ: the MLX conversion stores them as FP8 E4M3 with an F32 scale per
row (engram-{2,12,22,32}.bin, 16.5 GB each), identical in both builds. The DFly2 drafter and the
vision tower are BF16, as released.

The conversion to MLX's layout, the Engram files and the drafter's head use Xylolabs' hy4_mlx
converter from Xylolabs/Hy4-flash-MLX-4bit
(revision e0dc371); the quantized weights themselves are our GPTQ.


What's in this repo

Path Size Files Contents
Hy4-flash-abliterated-MLX-256GB/ 236.5 GB 109 the 256 GB Mac build
Hy4-flash-abliterated-MLX-128GB/ 162.1 GB 81 the 128 GB Mac build
LICENSE Tencent Hy License, from the base model

Each folder is a complete model directory:

File Contents
model-*.safetensors, model.safetensors.index.json the quantized text model (83 or 55 shards)
config.json the model config, with the per-module quantization, the Engram file layout and model_file: hy4_mlx/entry.py
engram-{2,12,22,32}.bin the Engram tables, FP8 E4M3 + a scale per row, read from disk
dfly.safetensors, dfly_head.safetensors the DFly2 drafter for multi-token prediction, and its BF16 copy of the output head
vision.safetensors, vision_config.json the vision tower, for hy4_mlx's optional image stage (not tested here)
tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json tokenizer, chat template, generation defaults
hy4_mlx/ the model code (Xylolabs, Apache-2.0, with LICENSE.transformers)
LICENSE Tencent Hy License

Usage (MLX)

mlx-lm has no Hy4-flash yet. Each folder ships the model code as hy4_mlx/, and config.json
names it as its model_file, so mlx-lm loads it with trust_remote_code. It was written for
mlx-lm 0.32.2.

pip install mlx-lm==0.32.2

# 128 GB Mac (for a 256 GB Mac, use Hy4-flash-abliterated-MLX-256GB)
hf download orcarouter/Hy4-flash-abliterated-MLX \
    --include "Hy4-flash-abliterated-MLX-128GB/*" --local-dir .

cd Hy4-flash-abliterated-MLX-128GB
python -m hy4_mlx.chat --model . --no-think --prompt "Write a haiku about the ocean."
python -m hy4_mlx.chat --model . --no-think          # interactive chat
python -m hy4_mlx.chat --model . --bench             # prompt and generation speed

Run hy4_mlx.chat from inside the folder: hy4_mlx is the folder next to the weights, not an
installed package. It decodes greedily with the DFly2 drafter (multi-token prediction, 5 draft
tokens); the trunk verifies every draft token, so the output is the same as plain greedy decoding.
--mtp-k 0 turns the drafter off, and --temp 0.9 samples (without the drafter).

From Python:

from mlx_lm import load, generate

model, tok = load("Hy4-flash-abliterated-MLX-128GB", trust_remote_code=True)
messages = [{"role": "user", "content": "Write a haiku about the ocean."}]
prompt = tok.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, reasoning_effort="no_think"
)
print(generate(model, tok, prompt=prompt, max_tokens=256))
  • Download the whole folder, including the four engram-*.bin files, with hf download as
    above. mlx-lm's own Hub download skips .bin files, and the model cannot run without them;
    hy4_mlx.chat checks them before loading.
  • Keep the folder on a fast internal SSD. The Engram tables are read from disk at every
    token.
  • Thinking. reasoning_effort is no_think, high or max (the template's default is
    high; --effort / --no-think in hy4_mlx.chat). The abliteration was fitted and evaluated
    with no_think.

Model details

Base model tencent/Hy4-flash, abliterated as orcarouter/Hy4-flash-abliterated
Architecture hy_v4_flash, in MLX through the bundled hy4_mlx. 42 layers: 1 dense + 41 latent-MoE (512 experts in a 1,536-wide latent, 16 active, 1 shared); 12 sparse + 30 sliding-window (512) MLA attention layers; 4 hyper-connection streams; Engram memory at four layers; hidden 3,072
Parameters 220B total (155.5B backbone, 63.4B Engram, 0.43B vision, 0.85B DFly2) · 11.3B active per text token · Vocabulary 120,832
Abliteration A 39-dimensional refusal subspace (the per-layer diff-of-means directions of layers 2–40) removed from 134 residual-stream writers, in BF16
Quantization GPTQ onto MLX's affine grid (group 64), per-expert Hessians; 128GB: routed experts 4-bit, the rest 8-bit; 256GB: 8-bit throughout. Engram tables FP8 E4M3 with per-row scales
Context 1,048,576 tokens (inherited; evaluated at 2K)
Runtime mlx-lm 0.32.2 with trust_remote_code, through the bundled hy4_mlx; text in, text out

Bias, risks, and limitations

  • Safety guardrails are substantially reduced, see the disclaimer, but not removed: on the
    BF16, 23–53% of harmful prompts are still judged refused.
  • Not yet run on Apple Silicon. The full builds have not been run on a Mac, so speed, peak
    memory and Metal-specific behaviour are unknown. The scores come from these weights
    dequantized into Transformers, and the hy4_mlx runtime was checked only on a 3-layer slice on
    MLX's CUDA backend.
  • The scores leave out the Engram tables' FP8 rounding, so the real KLD of these files is
    somewhat higher than shown.
  • The 128GB build is noticeably degraded. 82% next-token agreement is a real cut.
  • The 256GB build is closer but not lossless. 89% next-token agreement, about what a stock
    8-bit GGUF reaches on this model, which itself loses a lot.
  • The benchmark suite was run on the BF16 in Transformers, not on these files.
  • Not measured on these files: refusal, downstream accuracy, long context, tool use, image
    input, the drafter's acceptance rate. All fidelity measurements are on chat text at 2K context.
  • Requires trust_remote_code until mlx-lm adds Hy4-flash. Apps that cannot pass it will not
    load these files.
  • Inherits the biases and limitations of Hy4-flash.

License

The Tencent Hy License, Version 1.0, inherited from the base model
tencent/Hy4-flash. Abliteration and quantization
do not change the license obligations. Section 10 applies to every recipient: operating a
Model-as-a-Service business on this model needs a separate written license from Tencent. See the
license for its terms.

The hy4_mlx/ model code in each folder is by Xylolabs, from
Xylolabs/Hy4-flash-MLX-4bit, under the
Apache License 2.0; it adapts the Hugging Face Transformers implementation of Hy4-flash, whose
license ships as hy4_mlx/LICENSE.transformers.


One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website · Models · GitHub · Discord · X

Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration