license: apache-2.0
base_model: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
tags:

• gguf
• qwen3
• uncensored
• mtp
• llama-cpp
• quantized
• multimodal
• vision
  model_type: qwen3
  quantized_by: lllabs
  model_size: 27B
  num_params: 27320697856

Qwen3.6-27B-AEON-Ultimate-Uncensored — Q6_K GGUF + MTP

GGUF Q6_K quantization of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 with MTP (Multi-Token Prediction) weights included.

Key Features

• Q6_K quantization — near-lossless quality, practical for consumer hardware
• MTP weights included (866 tensors) — enables speculative decoding via --spec-type draft-mtp in llama.cpp for significantly faster inference
• Vision supported — mmproj file (931MB) available in separate repo for multimodal image input

Model Details

Quantization Process

1. BF16 original → FP16 GGUF conversion (V100 does not support BF16)
2. FP16 GGUF → Q6_K quantization with MTP weights preserved

How to Use

llama.cpp (recommended)

# Text-only
llama-server \
  -m qwen3.6-27b-Q6K-mtp.gguf \
  -ngl 99 \
  -c 80000 \
  --host 0.0.0.0 \
  --port 8080 \
  --spec-type draft-mtp

# With vision (download mmproj from https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF)
llama-server \
  -m qwen3.6-27b-Q6K-mtp.gguf \
  --mmproj qwen3.6-27b-mmproj.gguf \
  -ngl 99 \
  -c 80000 \
  --host 0.0.0.0 \
  --port 8080 \
  --spec-type draft-mtp

MTP (Multi-Token Prediction)

MTP enables speculative decoding without a separate draft model. The MTP weights are baked into this GGUF file. Use --spec-type draft-mtp to activate.

Performance

Tested on V100 SXM2 32GB:

Files

Notes

• V100 GPUs do not support BF16 — this model was converted via FP16 intermediate
• For single-GPU deployment, use --split-mode none -mg 0 to keep all weights on one GPU
• KV cache of 80K tokens fits within a single V100 32GB alongside the model

Credits

• Original model: AEON-7 — Qwen3.6-27B-AEON-Ultimate-Uncensored
• Base architecture: Qwen — Qwen3.6-27B
• Quantization tooling: llama.cpp