license: apache-2.0
base_model: OpenMOSE/Qwen3.5-REAP-262B-A17B
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- abliterated
- uncensored
- imatrix
- moe
- reap
- qwen3.5
- not-for-all-audiences
Qwen3.5-REAP-262B-A17B - Abliterated GGUF
Abliterated GGUF quant ladder of
OpenMOSE/Qwen3.5-REAP-262B-A17B,
itself a 34% REAP expert-pruning of Qwen3.5-397B-A17B down to
262B total / ~17B active. The IQ rungs are importance-matrix (imatrix) weighted.
Provenance chain: the abliterated bf16 safetensors base was converted to a Q8_0 master
(277.7 GB, near-lossless), and every rung here is cut from that master. The bf16 base lives at
RobinsonLabs/Qwen3.5-REAP-262B-A17B-abliterated,
use that repo if you want to re-abliterate, merge a LoRA, fine-tune, or roll your own quants.
Disclosure
This model is abliterated: the hard-refusal reflex on adult / creative content has been
reduced via single-direction weight orthogonalization. Harm guardrails are retained by design,
self-harm prompts still redirect to help (e.g. 988), and it is not intended to assist genuine
wrongdoing. Capability is preserved. Tagged not-for-all-audiences. Use responsibly, you are
responsible for what you generate with it. License inherited from the base model: Apache-2.0.
Files
| File | Quant | bpw | ~Size | imatrix | Notes |
|---|---|---|---|---|---|
...-Q5_K_M.gguf |
Q5_K_M | 5.68* | ~186 GB | no | highest-fidelity rung published |
...-Q4_K_M.gguf |
Q4_K_M | 4.85* | ~159 GB | no | K-quant quality pick |
...-Q4_K_S.gguf |
Q4_K_S | 4.55* | ~149 GB | no | |
...-IQ4_XS.gguf |
IQ4_XS | 4.28 | ~140 GB | yes | quality/size sweet spot |
...-Q3_K_M.gguf |
Q3_K_M | 3.83 | ~125 GB | no | |
...-IQ3_M.gguf |
IQ3_M | 3.51 | ~115 GB | yes | |
...-IQ3_XS.gguf |
IQ3_XS | 3.29 | ~107 GB | yes | |
...-IQ2_M.gguf |
IQ2_M | 2.62 | ~86 GB | yes | |
...-IQ2_XS.gguf |
IQ2_XS | 2.36 | ~77 GB | yes | |
...-IQ2_XXS.gguf |
IQ2_XXS | 2.12 | ~69 GB | yes | smallest |
bpw figures are as reported by llama-quantize, not nominal. The three starred rungs predate the
surviving build logs, their bpw is computed from exact file bytes over the 261.6B parameter count.
The IQ rungs are imatrix-weighted and land meaningfully smaller than the K-quant of comparable
quality: IQ4_XS undercuts Q4_K_S by ~9 GB, and the IQ2 family is the only path under 90 GB.
The chart shows the contested 2 to 5 bpw band; the higher-fidelity Q5_K_M rung is in the table
above.
Architecture notes
qwen3_5_moe hybrid: 60 decoder layers (45 linear-attn / DeltaNet + 15 full-attn,full_attention_interval=4), 333 experts with 10 active per token, hidden size 4096,
head dim 256, 262144 native context. No MTP / NextN layer. This is the text path only
(no vision mmproj).
The high expert count is the defining feature of this REAP tier: 333 experts versus 267 on the
48%-pruned 212B sibling.
More experts retained means more of the 397B parent's routing diversity survives, at the cost of
size.
Method
- Abliteration: single-direction weight orthogonalization (FailSpy / Labonne method). For every
matrix that writes the residual stream (o_proj, DeltaNetout_proj, fused expertdown_proj,
shared-expertdown_proj, and the token embedding), the rank-1 component along the refusal
direction is subtracted. Routers and norms pass through byte-identical. - Refusal direction, massive-activation guarded. The direction is captured with a
mean-difference control vector, then guarded against attention-sink contamination: the sink
dimensions that dominate raw activation magnitude (and would brick the model if ablated) are
detected across layers and excluded, and the direction is taken from the clean, spread-out
consensus of the late layers rather than a single sink-dominated layer. - Quant: convert and quantize with llama.cpp
(buildb9244). bf16 to a Q8_0 master (8.51 bpw as measured), then every rung cut from that
master. - imatrix: the IQ rungs are weighted by an importance matrix computed over the abliterated
model itself againstcorpus-rldomain, a domain-calibrated corpus. 200 chunks atn_ctx=512
(~102K tokens), 765 importance entries, final PPL 19.01 on the calibration set. The high chunk
count is deliberate: with 333 experts, a short calibration run leaves rarely-routed experts
under-exercised, and per-tensor coverage was still climbing well past the point where a
200-expert model would have saturated.
bf16 base
The full-precision bf16 safetensors master this ladder derives from is at
RobinsonLabs/Qwen3.5-REAP-262B-A17B-abliterated.
That repo is the master for further surgery (re-abliteration, LoRA merge, fine-tune) and for making
your own quants.
Provenance
Qwen3.5-397B-A17B (Apache-2.0) -> OpenMOSE/Qwen3.5-REAP-262B-A17B (34% REAP prune) ->
abliterated (bf16 master) -> Q8_0 master -> quant rungs. Every published rung, K and IQ alike,
is cut from the Q8_0 master (the IQ rungs with --allow-requantize), not directly from the bf16.
Recipe and diagnosis are Robinson Labs internal (WI #1423).
Built by Robinson Labs.