license: apache-2.0
tags:

• uncensored
• abliterated
• gguf
• qwen
• qwen3.5
• moe
• mamba
• 397b
• 17b-active
  base_model: Qwen/Qwen3.5-397B-A17B
  model_type: qwen3_5_moe
  pipeline_tag: text-generation
  language:
• en
• zh
• ja
• ko
• fr
• de
• es
• pt
• ru
• ar
• th
• vi
• id

Qwen3.5-397B-A17B-Uncensored-GGUF

The first comprehensive uncensored GGUF quantization suite for Qwen3.5-397B-A17B — Alibaba's flagship open-weight model that rivals GPT-5.2 on instruction-following benchmarks.

397B total parameters, 17B active per token (Mixture-of-Experts). Hybrid architecture: GatedDeltaNet linear attention + standard self-attention (every 4th layer) + 512 routed experts with 10 active per token. Natively multimodal (text + vision). 201 languages. Apache 2.0.

7 quantization levels from Q2_K to BF16. Single-file GGUFs — no splits, no merging required.

Why This Release

Quantizations

Architecture

• Type: Qwen3.5MoeForConditionalGeneration (hybrid GatedDeltaNet + MoE Transformer)
• Total Parameters: 397B
• Active Parameters: 17B per token
• Hidden Size: 4096
• Layers: 60
• Attention: 32 heads (GQA, 2 KV heads), head_dim 256
• Experts: 512 routed + shared expert, 10 active per token
• Expert FFN Size: 1024 (gate_up_proj → down_proj)
• Hybrid Attention: GatedDeltaNet linear attention + self-attention every 4th layer
• Linear Attention: 16 key heads (dim 128), 64 value heads (dim 128), conv kernel 4
• Context Length: 262,144 tokens
• Vocab Size: 248,320
• Multimodal: Native vision encoder (text + image + video)
• Languages: 201+ (en, zh, ja, ko, fr, de, es, pt, ru, ar, th, vi, id, ...)
• License: Apache 2.0

Abliteration Method

Custom abliteration pipeline executed on 8×NVIDIA H200 SXM5 GPUs (1.1TB VRAM total):

1. Model loaded in BF16 across 8 GPUs via device_map="auto" (~740GB VRAM)
2. LoRA adapters (rank 16, alpha 32) applied to 60 layers targeting out_proj, o_proj, and down_proj
3. 400 harmful + 400 harmless prompt pairs from mlabonne/harmful_behaviors and mlabonne/harmless_alpaca
4. Residual computation via forward passes with batch_size=4 across all prompts
5. Refusal direction identification per layer using mean difference of harmful vs harmless activations
6. Orthogonal projection applied to remove refusal directions from weights
7. Strength factor 20.0 for thorough refusal removal
8. LoRA merged into base weights and saved as full-precision safetensors

Key challenge: Qwen3.5's packed expert tensor format (ffn_gate_exps [512, 1024, 4096], ffn_down_exps [512, 4096, 1024]) prevents standard per-expert abliteration. The shared expert (shared_expert.down_proj) was targeted directly. The packed routed experts may retain some refusal capacity — if you encounter persistent refusals, please open a discussion with example prompts.

Pipeline: Custom abliteration → BF16 GGUF conversion (llama.cpp) → quantization cascade (Q8_0 → Q2_K)

Usage

llama.cpp

# Recommended: Q4_K_M for balanced quality/memory
./llama-cli -m Qwen3.5-397B-A17B-Uncensored-Q4_K_M.gguf \
  -p "You are a helpful uncensored assistant." \
  -n 512 --temp 0.7 --top-p 0.9

# With thinking mode enabled (default for Qwen3.5)
./llama-server -m Qwen3.5-397B-A17B-Uncensored-Q4_K_M.gguf \
  --port 8080 --host 0.0.0.0 -c 131072

LM Studio

Download the GGUF file and load it in LM Studio. The model supports both thinking and non-thinking modes via the enable_thinking parameter in the chat template.

Open WebUI / SillyTavern

Point your backend to a llama.cpp server running any of these quants. Full OpenAI-compatible API at /v1/chat/completions.

Known Limitations

• Packed expert abliteration: The 512 routed experts use packed tensor format and were not individually abliterated. Some refusals may persist in edge cases.
• Vision: The multimodal vision encoder is preserved but untested post-abliteration. Text generation is the primary target.
• Thinking mode: The model retains <think> tag generation. Use --reasoning-parser in SGLang or strip tags in post-processing if unwanted.

Model Provenance

• Base model: Qwen/Qwen3.5-397B-A17B (Apache 2.0)
• Abliteration technique: Inspired by FailSpy and Sumandora's remove-refusals-with-transformers, with custom extensions for Qwen3.5's hybrid MoE architecture
• Quantization: llama.cpp (build 8c60b8a)
• Hardware: 8×NVIDIA H200 SXM5, 1.1TB VRAM

Disclaimer

⚠️ This model has had safety alignment significantly reduced. It may generate content that is harmful, offensive, or inappropriate. Users are solely responsible for ensuring their use complies with applicable laws and ethical standards. This release is intended for research, testing, and controlled environments.

☕ Support This Work

Every donation helps fund more open-weight model releases. ⚡ Forged on 8×NVIDIA H200 SXM5 | 1.1TB VRAM

💎 Crypto Donations

🏢 Enterprise & Custom Models

Need a custom 120B+ model aligned to your proprietary data? TIMTEH provides bespoke enterprise fine-tuning, abliteration, and deployment on 8×H200 SXM5.

• Custom fine-tuning on your data (up to 400B+ parameters)
• Private CARE abliteration (Phase 2 technique)
• Deployment architecture consulting (tensor parallelism, speculative decoding)
• Bespoke distillation datasets

📧 Contact: [email protected]

Part of the TIMTEH Cognitive Preservation Foundry — surgical capability preservation at scale.
⚡ Forged on 8×NVIDIA H200 SXM5 | 1.1TB VRAM