For privacy reasons a browser tells us at most
"≥ 8 GB RAM, 8 cores" - same reading whether you have 8 GB
or 128 GB. It has no idea how much RAM is free right now, which
apps are open, or whether you have a GPU.
The Abliteration app is integrated with your machine
It reads your exact RAM, GPU model and VRAM,
free memory right now, and picks the sharpest quant that
still fits. Every model page lights up precisely for your rig.
And you can chat with any model, right now
The app is a full local runtime - no API keys, no subscription,
everything runs on your machine. Click any model on this site and
start a conversation in seconds.
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
This repository hosts ik_llama.cpp-compatible GGUF quantizations of MiMo-V2.6-Distill-Qwen-9B, abliterated with the ARA method. The parent is a 9-billion-parameter agentic model built by Xiaomi MiMo via supervised fine-tuning of Qwen3.5-9B on MiMo-generated data spanning software engineering, terminal tool-use, visual coding, and cybersecurity.
Architecturally, it inherits the Qwen3.5 hybrid attention stack: 24 gated DeltaNet linear-attention layers interleaved with 8 full-attention layers across 32 blocks (full attention every 4th layer). The linear layers carry a fixed-size recurrent state instead of a growing KV cache, which keeps memory growth flat at long context. A vision encoder feeds image tokens into the same text backbone.
Quantizing a hybrid stack is not the same as quantizing a dense transformer. The delta-net recurrence parameters (conv1d, ssm_a, ssm_dt.bias) are held at full precision in every build here — they are small tensors with outsized influence on state drift, and quantizing them breaks long-context coherence before it shows up in perplexity.
MaxOptima Custom Quant
This release includes MaxOptima, a layer-aware mix built for the Qwen3.5 delta-net geometry. The strategy splits into two tracks: the SSM / linear-attention path keeps its recurrence and normalization tensors at F32, while the 8 full-attention layers (3, 7, 11, 15, 19, 23, 27, 31) — the only layers that carry a real KV cache — get higher-precision projections because they are the ones holding the long-range context together.
MaxOptima strategy: Embeddings and output locked at Q6_K. All delta-net recurrence and norm tensors (conv1d, ssm_a, ssm_dt.bias, attn/ssm/output norms) held at F32. Boundary blocks 0 and 31 get protected SSM-out, QKV, gate, and FFN tensors. Full-attention layers get Q6_K on V and output, Q5_K on Q and K. Mid-stack SSM-out and QKV use IQ5_K. Gates compress to IQ3_KS; FFN up/down to IQ4_KSS. Default fallback is Q5_K.
Custom layer-aware mix. F32 on the delta-net recurrence path, Q6_K full-attention V/output, IQ5_K mid-stack SSM-out and QKV, IQ3_KS gates, IQ4_KSS FFN. The recommended build for this model.
Standard IQK // High & Mid Fidelity
IQ5_K
Higher fidelity 5-bit. Best quality-to-size for systems with sufficient memory.
IQ4_K
Standard 4-bit IQK. Uses 256-value super-blocks for scaling.
IQ4_KS
"Small" 4-bit variant with tighter packing. Lower memory footprint.
IQ4_KS_R4
Row-interleaved variant of IQ4_KS, optimized for ik_llama.cpp's matmul kernels.
Trellis // Alternative 4-Bit
IQ4_KT
Dynamic programming trellis quant. Uses a fundamentally different error-minimization strategy than IQK.
Abliteration Report // ARA Method
The base abliteration uses Arbitrary-Rank Ablation (ARA) via Heretic, across layers 7 through 29 — the middle-to-late blocks where refusal directions concentrate in the Qwen3.5 hybrid stack. The run settled on trial 18 of 40, with a measured refusal rate of 42/100 and a KL divergence of 0.3377.
Run metrics, not guarantees. A 42% refusal rate means refusals are reduced, not eliminated. KL divergence of 0.3377 is moderate — validate capability retention on your own tasks. These quants preserve the abliterated weights as-is; behaviour is inherited from the parent checkpoint.
View Full ARA Parameters
start_layer_index
7
end_layer_index
29
preserve_good_behavior_weight
0.5826
steer_bad_behavior_weight
0.0465
overcorrect_relative_weight
0.8496
neighbor_count
9
trial
18 / 40
refusals
42 / 100
KL divergence
0.3377
Imatrix Specification
This release uses the V3 Calibration Dataset used across previous Austriani releases. Calibration ran at 500 cycles per chunk with a 1024-token context window, processing approximately 1000 chunks.
Compatibility
These GGUF files are built exclusively for ik_llama.cpp. IQK, Trellis, and MaxOptima quants will not load in mainline llama.cpp. The hybrid delta-net architecture also requires a build recent enough to carry Qwen3.5 linear-attention support — older ik_llama.cpp builds will fail on the SSM blocks regardless of quant format.
Vision: the multimodal projector is included, but image input through ik_llama.cpp depends on your build's mmproj support for this architecture. Test before relying on it.
README history
3 versions
The author's README evolved over time. Click a version to see its content at that point.
2026-11-13Update README.md850de5c14.8 KB
Loading...
2026-11-13Update README.mdec9430c14.8 KB
Loading...
2026-11-13initial commitee4694728 B
Loading...
Catalog is the map. Apps are the tools.
Run models on your own machine, not in the cloud.
Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.