base_model: eggdog100/Qwen3.6-35B_Zenith
license: cc-by-nc-4.0
language:

• en
• zh
  library_name: transformers
  pipeline_tag: image-text-to-text
  tags:
• abliterated
• uncensored
• moe
• multimodal
• qwen3_5_moe
• abliterix

Qwen3.6-35B Zenith — Abliterated

An abliterated (refusal-suppressed) derivative of
eggdog100/Qwen3.6-35B_Zenith.
Refusal behavior is reduced while the model's capabilities, coherence, and multimodal
vision are preserved. The edit is deliberately minimal and fully verified (see below).

Lineage: Qwen/Qwen3.6-35B-A3B → Zenith (LoRA capability-SFT on public data) → this (abliterated).

At a glance

Method — the deliberately-minimal "o_proj-only" recipe

The goal was to suppress refusals without touching anything that could destabilize this
hybrid-MoE model (the failure mode where editing MoE routing or the recurrent GatedDeltaNet
state collapses the model into repetition).

• Edited: only the 10 full-attention self_attn.o_proj layers (indices 3,7,…,39),
  via a norm-preserving orthogonal projection of the refusal direction, gaussian-decay
  strength concentrated in the mid/late layers.
• Left bit-identical to the base: the 30 GatedDeltaNet linear_attn.out_proj layers,
  the MoE router, all 256 experts, the vision tower, and the MTP head.
  (A source patch isolated linear_attn.out_proj into its own component so the recurrent SSM
  state is never perturbed.)
• steering_mode = direct — a static weight edit that bakes into the weights (no runtime
  forward hooks; the operating point that was validated is exactly the one that ships).
• Optimized by Optuna (100 trials) to minimize KL subject to refusals ≤ 10/100, scored by a
  local 72B LLM judge (so degenerate/garbled outputs are rejected, never selected).

Verification — every claim measured, nothing assumed

1. Weight diff vs. base — exactly the 10 full-attn o_proj tensors changed
   (max|Δ| ≈ 0.025–0.068); the 30 GatedDeltaNet out_proj, 40 routers, 256 experts,
   333 vision tensors, and the MTP head are byte-identical.
2. Search ↔ export edit equivalence — the vLLM-search and HF-export static o_proj edits were
   proven mathematically equivalent (up to floating-point rounding).
3. Reloaded-artifact coherence — the exported model was reloaded and confirmed coherent and
   correct on math, code, translation, knowledge, and multi-step reasoning — 0 repetition
   collapses.

Usage

Thinking model — use Qwen sampling (temperature=0.6, top_p=0.95, top_k=20); avoid greedy
decoding and large repetition/presence penalties.

from vllm import LLM, SamplingParams
llm = LLM("eggdog100/Qwen3.6-35B_Zenith-Abliterated", dtype="bfloat16",
          gpu_memory_utilization=0.9, max_model_len=16384, trust_remote_code=True)
tok = llm.get_tokenizer()
msgs = [{"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "..."}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = llm.generate([prompt], SamplingParams(temperature=0.6, top_p=0.95, top_k=20, max_tokens=4096))
print(out[0].outputs[0].text)

Requires a recent transformers (≥ 5.12) and vllm with native qwen3_5_moe support.

Quantizations (GGUF)

In gguf/, converted with convert_hf_to_gguf.py --no-mtp (the multi-token-prediction
head is excluded — otherwise llama.cpp fails to load with missing tensor 'blk.40…'):

The Qwen3.6 hybrid (GatedDeltaNet + MoE) is newly supported in llama.cpp; these load and
generate correctly, but expect lower throughput than mature architectures until upstream
kernels mature. For full-speed serving, use the bf16 weights via vLLM / transformers.

Acknowledgments

• Abliteration tool: abliterix by
  Wangzhang Wu — a derivative of Heretic by
  Philipp Emanuel Weidmann. The o_proj-only / GatedDeltaNet-isolation / MoE-untouched
  recipe and the local-LLM-judge optimization used here are all built on abliterix.
• Base model: eggdog100/Qwen3.6-35B_Zenith.
• Architecture: Qwen3.6 / qwen3_5_moe by the Qwen team, Alibaba Group.
• Quantization: llama.cpp (--no-mtp).

Provenance & License

This is a derivative of eggdog100/Qwen3.6-35B_Zenith
(© its author), which is itself a LoRA capability-SFT of Qwen/Qwen3.6-35B-A3B (Apache-2.0)
trained on openly-licensed public data.

License: CC-BY-NC-4.0 (non-commercial) — inherited from Zenith. Zenith's training set
includes HuggingFaceH4/no_robots and Estwld/empathetic_dialogues_llm, both CC-BY-NC, so
the resulting weights (and this derivative) carry a non-commercial restriction. Attribution
to the base model and the Qwen team is preserved.

Responsible use

This model has reduced refusal behavior. You are solely responsible for lawful and ethical
use. Released for research/personal use. No warranty.