license: apache-2.0
language:
- en
- zh
base_model: Youssofal/Qwen3.6-27B-Abliterated-Heretic-Uncensored-BF16
pipeline_tag: image-text-to-text
tags: - mlx
- oq
- 4bit
- mixed-precision
- uncensored
- abliterated
- qwen
- apple-silicon
Qwen3.6-27B Abliterated Heretic Uncensored — oQ4
Mixed-precision quant of Youssofal/Qwen3.6-27B-Abliterated-Heretic-Uncensored-BF16, produced with oQ (oMLX v0.5.4). Vision tower intact — image + video input still works. Standard MLX safetensors — compatible with oMLX, mlx-lm, LM Studio, and any MLX-capable app.
What is oQ?
Unlike uniform 4-bit quantization, oQ is a data-driven mixed-precision quantizer that calibrates per-layer sensitivity and allocates bits where they matter most. Critical layers (embeddings, LM head, the most sensitive transformer layers) are automatically promoted to 8-bit, while less sensitive layers stay at 4-bit. Typical result: ~4.6 bits-per-weight.
Benchmarked on Qwen3.5-35B-A3B (oMLX project):
| Benchmark | mlx-lm 4-bit | oQ4 |
|---|---|---|
| MMLU (300) | 79.7% | 83.3% |
| TruthfulQA (300) | 87.7% | 88.0% |
| HumanEval (full) | 87.2% | 85.4% |
| MBPP (300) | 71.7% | 74.3% |
Performance (oMLX on M4 10-core)
| Context | PP tok/s | TG tok/s | Peak Mem |
|---|---|---|---|
| 1k | 63.6 | 6.5 | 16.4 GB |
| 4k | 59.9 | 6.3 | 17.8 GB |
| Batch | TG tok/s | Speedup |
|---|---|---|
| 1× | 6.5 | 1.00× |
| 2× | 11.6 | 1.78× |
| 4× | 24.4 | 3.75× |
Why this quant
The original BF16 weights require ~55 GB. This oQ4 quant runs in ~16–18 GB on Apple Silicon.
Quick start
# oMLX
omlx serve --model underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4
# mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
Original model
- Base: Qwen/Qwen3.6-27B
- Abliterated: Heretic-style two-stage MPOA pipeline, magnitude-preserving refusal attenuation (KL 0.0282)
- Vision: Full multimodal — image + video input supported
License
Apache 2.0, inherited from base model.