base_model:
- Qwen/Qwen3.5-122B-A10B
- OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated
pipeline_tag: text-generation
library_name: transformers
tags: - qwen3_5_moe
- image-text-to-text
- conversational
- gguf
- moe
- agent
- moe
- abliterated
- uncensored
- sft
- dpo
- opus
- qwopus
- kimi
- kimi-k2
- distill
- multimodal
- vision
- mtp
Qwopus3.5-122B-Distill-Kimi-Uncensored-GGUF
This repository contains the GGUF quantized files for OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated.
- Original Model: OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated
- Architecture: Qwen3.5-122B-A10B
- License: Apache 2.0
- Vision: works as expected (image / video → text).
- MTP: the head is present and shape-compatible, but in our testing it produced no measurable speedup or quality gain on this checkpoint. It is shipped intact for completeness and forward-compatibility, but would need to be retrained to be useful — happy to do so if there is interest in the model.
| Quant Type | Size | Description |
|---|---|---|
| IQ2_M | 55-59 GB | Mixed Precision for Better Quality |
| IQ3_M | 62-67 GB | Mixed Precision for Better Quality |
| IQ4_NL | 76-81 GB | Mixed Precision for Better Quality |
Overview
The pipeline:
- Refusal Ablation — Residual-stream refusal directions (one per decoder layer, layers 19–45) were extracted via diff-in-means on a labeled prompt set and baked into the weights as a per-matrix delta — see the abliterix framework for the methodology.
- Healing — Stage A: Constrained-LoRA SFT on Opus reasoning data — Supervised finetuned on a curated set of Claude Opus reasoning traces (single-turn, ~8k rows). To keep the abliteration mathematically intact during training, a custom orthogonality projection is applied to every LoRA
B-matrix on residual-write modules after each optimizer step (B := B − r·(rᵀB)), so the LoRA update is forbidden from re-introducing the refusal direction. LoRA rank 32, α 64, 54 protected modules across 27 decoder layers. Verified residual after training:max ‖rᵀB‖₂ = 8.5 × 10⁻¹⁰. - Healing — Stage B: Unconstrained SFT on chosen completions — A second short SFT pass (LoRA r=16, α 32, no orthogonality constraint) on the chosen answers (including reasoning chains) from an internal preference dataset, to tighten on the deployment distribution and remove the last bits of drift introduced by Stage A.
- Kimi K2.6 Reasoning DPO — A targeted preference-optimization pass distilled from Kimi K2.6 to improve reasoning verbosity and eliminate degenerate looping. See the dedicated section below.
- Vision + MTP Restoration — The original Qwen3.5 vision tower (333 tensors, depth 27, hidden 1152) and MTP head (785 tensors, 1 hidden layer) were grafted back from the upstream
Qwen/Qwen3.5-122B-A10Bshards. Tensor names, shapes, andconfig.jsonschema (Qwen3_5MoeForConditionalGeneration,model_type: qwen3_5_moe) match the base model exactly — so this checkpoint loads anywhere the original loads.
Key Properties:
- Uncensored across the standard refusal axes
- Reasoning preserved and improved (Opus-style think-then-answer + Kimi K2.6 reasoning DPO)
- Fewer looping / repetition failures on long conversations
- Multimodal: vision (image / video) and MTP heads carried forward
- Drop-in shape compatibility with
Qwen/Qwen3.5-122B-A10B
Kimi K2.6 Reasoning DPO
On top of the base abliteration + Opus healing, this release adds a focused healing pass built from Kimi K2.6:
- ~3,000 samples distilled from Kimi K2.6 were used for DPO (Direct Preference Optimization), alongside synthetic datasets also generated from Kimi K2.6.
- Improved reasoning verbosity — the model now produces more complete, better-structured reasoning on the ~12% of requests where the previous release tended to under-explain or cut its chain-of-thought short.
- Fixed looping / repetition — degenerate loops that appeared on 2–6% of long-tail conversations (long context, multi-turn) were largely eliminated.
The DPO pass targets the language model's reasoning behavior only; the abliteration, vision tower, and MTP head are unchanged by this step.
Evaluation
This model family outperforms the full-precision (BF16) Qwen/Qwen3.5-122B-A10B baseline across reasoning, coding, and tool-use benchmarks:
| Benchmark | Qwen3.5-122B-A10B (BF16, baseline) | Qwopus3.5-122B-A10B |
|---|---|---|
| CTI | 64.8 | 71.5 |
| LiveCodeBench | 78.9 | 79.9 |
| BFCL | 72.2 | 85.6 |
BFCL is the Berkeley Function-Calling Leaderboard (tool use); LiveCodeBench is contamination-controlled code generation.
Notes
- License: Other (inherits from the Qwen3.5 base license)
- Base Model: Qwen/Qwen3.5-122B-A10B
- Healing: Opus reasoning SFT + Kimi K2.6 reasoning DPO (≈3,000 distilled samples + synthetic data)
- Modality: Text + Vision (image / video) + MTP
- Architecture: Qwen3 MoE (~10B active / 122B total) + Qwen3-VL vision tower + MTP head
Thanks
- Jackrong — for the idea of Qwopus merges (Opus distillations on Qwen models).
- wangzhang — for the wonderful abliterix framework, which was customized to do this abliteration.
Disclaimer
Use is the responsibility of the user. Ensure your usage complies with applicable laws, platform rules, and deployment requirements.
How to Use
These GGUF files are fully compatible with llama.cpp and popular graphical interfaces like LM Studio.
using llama.cpp CLI:
./llama-cli -m /path/to/model/Qwopus3.5-122B-Distill-Kimi-IQ3_M.gguf \
-p "Hello, how are you?" \
-sys "You are a helpful AI" \
-n 4096 \
-c 8192
using llama-server :
./llama-cli -m /path/to/model/Qwopus3.5-122B-Distill-Kimi-IQ3_M.gguf \
--host 0.0.0.0 \
--port 8080