license: apache-2.0
base_model: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic
base_model_relation: quantized
pipeline_tag: text-generation
tags:

• awq
• int4
• w4a16
• moe
• qwen3_5_moe
• uncensored
• abliterated
• compressed-tensors
• llm-compressor
• quantized
  language:
• en

Qwen3.6-35B-A3B-uncensored-heretic-AWQ (W4A16)

AWQ 4-bit (W4A16) quantization of
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic
— an uncensored ("heretic"/abliterated) variant of Qwen3.6-35B-A3B
(qwen3_5_moe: a hybrid GatedDeltaNet (linear-attention) + full-attention
Mixture-of-Experts with 256 routed experts (8 active) + a shared expert, plus a
vision tower and an MTP head).

Variant: AWQ W4A16 — 4-bit symmetric integer weights, group size 128, with
activation-aware scaling. Activations stay BF16.
Disk size: ~21.8 GB (vs ~70 GB BF16, ~3.2×)
Quantized by: sahilchachra
Tooling: llm-compressor 0.12 (AWQModifier + QuantizationModifier) → compressed-tensors pack-quantized

Why this source: the popular HauhauCS/Qwen3.6-35B-A3B-Uncensored-* repos ship
only GGUF (no full-precision weights), which cannot be honestly AWQ-quantized. This
checkpoint quantizes a genuine BF16 uncensored Qwen3.6-35B-A3B instead.

What is quantized

Quantized to int4 (30,880 modules):

• routed experts mlp.experts.{0..255}.{gate,up,down}_proj on all 40 layers — 30,720
• shared expert mlp.shared_expert.{gate,up,down}_proj on all 40 layers — 120
• full-attention self_attn.{q,k,v,o}_proj on the 10 full-attention layers — 40

Kept in BF16 (not quantized): the GatedDeltaNet linear_attn (mamba) layers,
the MoE router (mlp.gate) and shared_expert_gate, the vision tower, the
MTP head, token embeddings, lm_head, and all norms.

Calibration

AWQ is calibration-based. Calibrated on 32 sequences × 512 tokens of general
instruction-following chat from
HuggingFaceH4/ultrachat_200k,
rendered through the model's chat template. Because this is the base instruct model
with refusals abliterated (not a task-specific fine-tune), general chat best matches
its inference distribution. During calibration every one of the 256 experts sees all
calibration tokens (moe_calibration_context), so per-expert AWQ scales are
well-estimated despite sparse 8/256 routing.

Loading / serving

This checkpoint stores experts in the per-expert compressed-tensors
(pack-quantized) layout that vLLM consumes for qwen3_5_moe. Serve with vLLM:

from vllm import LLM, SamplingParams
llm = LLM(model="sahilchachra/Qwen3.6-35B-A3B-uncensored-heretic-AWQ",
          trust_remote_code=True, dtype="bfloat16")
print(llm.generate(["Write a quicksort in Python."], SamplingParams(max_tokens=256))[0].outputs[0].text)

Notes

• Text-only: the source repo nests its vision-tower weights under a non-standard
  key prefix (model.language_model.visual.* instead of model.visual.*), so the
  vision tower does not load functionally. This is a quantization of the language
  model; treat it as a text-generation model.
• GatedDeltaNet kernels (causal-conv1d / flash-linear-attention) fall back to a
  pure-PyTorch path where unavailable (e.g. ARM/Jetson) — correct, just slower.
• compressed-tensors == 0.17.1's eager decompress_model mis-reads group_size
  for pack-quantized checkpoints; load via vLLM (its own loader) or a fixed
  compressed-tensors. The weights/scales on disk are standard and correct.

Uncensored

This is an abliterated / uncensored model (refusal directions removed). It will
follow instructions it would otherwise decline. Use responsibly and in compliance with
applicable law and the Apache-2.0 / Qwen terms.

Original model

See llmfan46/Qwen3.6-35B-A3B-uncensored-heretic
and base Qwen/Qwen3.6-35B-A3B for
architecture, intended use, and license.