library_name: mlx
tags:
- mlx
- oq
- quantized
base_model: - LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-GGUF
Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-dequantized-oQ8e-mtp
This model was quantized using oQ (oMLX v0.5.4rc2) mixed-precision quantization.
Quantization details
- Model type: qwen3_5_moe
- Bits: 8
- Group size: 64
- Format: MLX safetensors
Performance Benchmark
- Run on: Apple Mac Studio M4 Max 128GB
- https://omlx.ai/benchmarks/2ozvcitk
oMLX - LLM inference, optimized for your Mac
https://github.com/jundot/omlx
Benchmark Model: Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-dequantized-oQ8e-mtp
Engine: Auto
================================================================================
Single Request Results
--------------------------------------------------------------------------------
Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem
pp1024/tg128 611.4 11.36 1674.8 tok/s 88.7 tok/s 2.061 558.9 tok/s 35.47 GB
pp4096/tg128 2302.9 11.59 1778.6 tok/s 87.0 tok/s 3.782 1116.8 tok/s 36.35 GB
pp8192/tg128 4800.0 11.92 1706.7 tok/s 84.5 tok/s 6.321 1316.1 tok/s 36.86 GB
pp16384/tg128 10530.7 12.51 1555.8 tok/s 80.6 tok/s 12.127 1361.6 tok/s 37.69 GB
pp32768/tg128 24548.2 13.72 1334.8 tok/s 73.4 tok/s 26.300 1250.8 tok/s 39.49 GB
pp65536/tg128 63110.8 16.14 1038.4 tok/s 62.4 tok/s 65.172 1007.6 tok/s 43.07 GB
pp131072/tg128 192255.0 19.96 681.8 tok/s 50.5 tok/s 194.803 673.5 tok/s 50.36 GB
pp200000/tg128 406278.2 24.34 492.3 tok/s 41.4 tok/s 409.382 488.9 tok/s 58.08 GB
Continuous Batching
pp1024 / tg128
--------------------------------------------------------------------------------
Batch tg TPS Speedup pp TPS pp TPS/req TTFT(ms) E2E(s)
1x 88.7 tok/s 1.00x 1674.8 tok/s 1674.8 tok/s 611.4 2.061
2x 146.9 tok/s 1.66x 659.2 tok/s 329.6 tok/s 3106.2 4.849
4x 208.0 tok/s 2.34x 1510.5 tok/s 377.6 tok/s 2599.6 5.009
8x 265.5 tok/s 2.99x 1493.4 tok/s 186.7 tok/s 5113.3 8.887
Intelligence Benchmark Comparison
Intelligence Benchmark Comparison
Mode Sampled Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-dequantized-oQ4e-mtp Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-dequantized-oQ8e-mtp
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MMLU Sample 1000/14042 87.8% 87.9%
TRUTHFULQA Full 817 80.9% 81.2%
HUMANEVAL Full 164 92.1% 94.5%
LIVECODEBENCH Sample 300/1055 48.7% 45.0%
--- Detail ---
Model: Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-dequantized-oQ4e-mtp
Benchmark Accuracy Correct Total Time(s) Think
--------------------------------------------------------------
MMLU 87.8% 878 1000 3798.1 Yes
TRUTHFULQA 80.9% 661 817 2633.4 Yes
HUMANEVAL 92.1% 151 164 1284.1 Yes
LIVECODEBENCH 48.7% 146 300 11636.8 Yes
Model: Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V6-dequantized-oQ8e-mtp
Benchmark Accuracy Correct Total Time(s) Think
--------------------------------------------------------------
MMLU 87.9% 879 1000 4115.5 Yes
TRUTHFULQA 81.2% 663 817 3382.5 Yes
HUMANEVAL 94.5% 155 164 1437.6 Yes
LIVECODEBENCH 45.0% 135 300 12955.1 Yes