license: apache-2.0
language:

• en
  base_model: Qwen/Qwen3.6-35B-A3B
  tags:
• abliterated
• uncensored
• qwen
• qwen3.6
• moe
• mixture-of-experts
• hybrid-attention
• gated-deltanet
• thinking-mode
• research
  pipeline_tag: text-generation

CyberNeurova · Qwen3.6-35B-A3B · Abliterated

CyberNeurova research — cyberneurova.ai.
Multi-axis abliteration on a hybrid-attention MoE chat model with thinking-mode.

A permanently-abliterated version of
Qwen/Qwen3.6-35B-A3B — Qwen's
35B-total / 3B-active mixture-of-experts model with hybrid attention (linear
Gated DeltaNet layers interspersed with standard self-attention) and
integrated chain-of-thought reasoning. The refusal direction was captured in
the LM residual stream and orthogonalized out of every write-to-residual
Linear (and the 3D fused expert tensor). Inference is unchanged — no runtime
hooks, no slowdown.

This is the most architecturally complex model in the CyberNeurova
abliteration line to date — combining four framework challenges in one:
MoE with 256 fused experts + shared expert, hybrid attention (linear_attn

• self_attn per block), 3-level attribute paths (model.model.language_model.layers),
  and CoT-wrapped thinking-mode refusals that needed a classifier upgrade
  just to be detected.

Headline results

Measured on the bf16 abliterated model via vLLM (baseline + ablated
back-to-back from the same prompt set, scored with our CoT-aware
refusal classifier):

Standouts:

• Refusal fully collapsed on both probes (90.9% → 0% and 85.5% → 0%) — no surviving refusal patterns even on out-of-distribution prompts
• Cyber unlock is substantial: +30 pp on cyber-weapons, +19.7 pp on hacking — these benchmarks score both compliance AND technical specificity, so the numbers reflect actually-useful security knowledge being unlocked, not just less hedging
• No capability tax — coding/coherence/reasoning all preserved or improved
• Diversity went up (+12.2 pp distinct-2) without coherence regression

A note on tool-calling: the 9-bench suite includes a tool-calling
benchmark that scored 0/0 on both baseline and ablated — this is a
grader artifact, not capability loss. The model wraps its tool
calls in <think>...</think> reasoning blocks; the grader regex
expects raw JSON. Both scoring zero (rather than baseline ~0.9 →
ablated 0) confirms it's the grader and not abliteration breaking
tool-calling. We're shipping a CoT-aware grader fix in the next
framework round.

See cyberneurova-qwen3.6-35b-a3b-abliterated.pdf and
qwen3.6-35b-a3b-abliterated.html
in this repo for the full visual benchmark report.

How it works

Capture: refusal direction extracted at layer 24 of 40 (the 0.6
fraction sits in a linear_attn cluster between two self_attn
anchors), method normalized_diff on AdvBench-style harmful prompts vs
Alpaca harmless prompts.

Ablation: weights orthogonalized against the captured direction
across ALL 40 layers, covering:

• Attention writes: linear_attn.out_proj on 30 Gated DeltaNet
  layers, self_attn.o_proj on 10 standard attention layers
• MoE writes: the fused experts.down_proj Parameter (shape
  [256, 2048, 512] — 256 expert weights orthogonalized in a single
  batched operation) + shared_expert.down_proj (standard Linear)
• Embedding: embed_tokens.weight orthogonalized in the hidden dim

This is the first multi-axis abliteration we've shipped — MoE + hybrid
attention + fused-experts + VLM-class attribute path. Each axis required
a small framework upgrade; details in our BUILD_NOTES.md (sections
§19-§20).

How to download

hf download cyberneurova/CyberNeurova-Qwen3.6-35B-A3B-abliterated \
  --local-dir ./Qwen3.6-35B-A3B-abl

~66 GB safetensors + tokenizer + processor.

How to run

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "cyberneurova/CyberNeurova-Qwen3.6-35B-A3B-abliterated",
    dtype="bfloat16", device_map="cuda:0",
    trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained(
    "cyberneurova/CyberNeurova-Qwen3.6-35B-A3B-abliterated",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Your prompt here."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

The model emits <think>...</think> chain-of-thought before its answer.
For most use cases you'll want max_new_tokens >= 512 to capture both
the reasoning and the response.

Hardware requirements

Tested on RTX PRO 6000 Blackwell (102 GB VRAM). Should work on any
80 GB+ data-center GPU. GGUF variants will broaden the hardware
floor once converted.

Intended use

Defensive security research, red-team evaluation baselines, study of
how refusal directions behave in MoE + reasoning-mode + hybrid-attention
architectures. Useful as a counterfactual against the original
Qwen/Qwen3.6-35B-A3B for measuring the behavioral impact of safety
RLHF on a thinking-mode model.

Not intended for automating harmful action. The abliteration removes
canonical refusal behavior but does not remove the model's underlying
knowledge — the model still recognises harmful instructions as harmful,
it simply no longer refuses them by pattern.

Limitations

• Tool-calling benchmark measurement is currently a grader artifact
  (0/0 both sides) — the grader expects raw JSON, the model wraps its
  tool calls inside <think> reasoning. A CoT-aware grader fix is
  scheduled for the next framework round; we expect real tool-calling
  capability to land near baseline once the grader is fixed.
• Coherence and perplexity preserved on wikitext-2, but extended
  thinking-mode behavior on very long (>4k tokens) reasoning chains is
  not exhaustively validated.
• The hybrid-attention layout means the refusal direction was captured
  through both linear_attn and self_attn blocks indistinguishably — we
  don't know yet whether the linear_attn layers contribute differently
  to the refusal feature than the self_attn layers do. That's a follow-up
  research question.
• GGUF quants (Q4_K_M / Q8_0) are not in this release — upstream
  llama.cpp does not yet support the qwen3_5_moe architecture.
  bf16 weights only for v1; GGUF will follow once upstream tooling
  catches up.

License

Apache 2.0 (inherits from upstream Qwen3.6-35B-A3B).

Acknowledgements

• Alibaba Qwen for Qwen3.6-35B-A3B
• Arditi et al. 2024 for the refusal-direction methodology

Related releases by CyberNeurova

• cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated
• cyberneurova/CyberNeurova-DeepSeek-V4-Flash-abliterated-GGUF
• cyberneurova/CyberNeurova-Lance-3B-abliterated