license: apache-2.0
base_model: Qwen/Qwen3.8-27B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
language:

• en
• zh
  tags:
• abliterated
• qwen
• qwen3
• qwen3.8
• gguf
• llama.cpp
• uncensored
• ai-red-team
• red-teaming
• vision-language
• mmproj
• mtp
• function-calling
• reasoning

Qwen3.8-27B-Uncensored-GGUF

GGUF quants (2-bit → 16-bit) of the abliterated (refusal-removed) Qwen3.8-27B — for llama.cpp

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · Model Card · GitHub · Ollama · Discord · X

GGUF conversions of Qwen3.8-27B-Uncensored
— an abliterated (refusal-removed) build of Qwen's Qwen3.8-27B, a 27B dense hybrid-attention
(Gated DeltaNet linear + full attention) native vision-language model with reasoning, tool-calling,
and an MTP speculative-decoding head. These files run in llama.cpp (CPU / CUDA / Metal / ROCm),
quantized from 2-bit to 16-bit, with a separate mmproj file that restores vision.
Browse all models in the OrcaRouter Model Catalog. Qwen3.8 27B is
deployed as API on OrcaRouter.

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing
the refusal direction out of the residual stream). It will comply with harmful, unethical, or illegal
requests the original Qwen3.8-27B would refuse. Released strictly for legitimate research —
interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You
assume full responsibility for how you use it and everything it generates; add your own safety and
moderation layers before any deployment. Use must comply with the
Apache 2.0 License inherited from the base model and all
applicable law. The authors accept no liability for misuse.

Requirements

• A recent llama.cpp built from source (the qwen35
  hybrid-GDN architecture and the MTP / nextn speculative head — merged 2026-05 — must be present).
  Older releases will not load these files.
• The GDN linear-attention layers are stored as SSM-style tensors (ssm_*); full-attention layers as
  attn_*; the MTP head as block nextn.* (qwen35.nextn_predict_layers).

Files

Standard K-quants

IQ quants (imatrix)

Lower-bit quants built with an importance matrix (computed on English + Chinese calibration
text) — better quality-per-bit than plain K-quants at the low end, especially IQ3/IQ2.

Vision

All quants (K-quant and IQ) preserve the MTP (nextn) head and the GDN hybrid architecture;
vision is provided by the separate mmproj file. The IQ files were quantized with an importance
matrix (computed on English + Chinese calibration text) for better low-bit fidelity; the matrix
itself is not shipped, as it is only needed to re-quantize these files, not to run them.

Usage (llama.cpp)

Download

hf download orcarouter/Qwen3.8-27B-Uncensored-GGUF \
  Qwen3.8-27B-Uncensored-Q4_K_M.gguf mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
  --local-dir ./qwen38-uncensored

Chat (text)

./llama-cli -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf --jinja -c 8192 -p "Hello!"

OpenAI-compatible server (tool calling + reasoning + vision)

./llama-server -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
  --host 0.0.0.0 --port 8000 -c 8192 --jinja

• Vision: pass --mmproj …, then send OpenAI image_url content parts (base64 data-URI or URL).
• Tool calling: --jinja enables the Qwen tool template; use standard OpenAI tools + tool_calls.
• Reasoning (thinking): thinking is on by default; toggle per request via
  chat_template_kwargs.enable_thinking. The reasoning trace is returned in reasoning_content
  (give max_tokens room, e.g. ≥ 2048, so the final answer isn't truncated by the thinking budget).
• MTP speculative decoding: the nextn head is embedded; enable llama.cpp's MTP/speculative path
  for a decode speed-up (optional — the model runs fine without it).

Or skip llama.cpp — run it with Ollama

The same quants are published as Ollama tags, with the mmproj projector already bundled in (no second
file, no --mmproj flag) and vision / tools / thinking wired up:

ollama run orcarouter/Qwen3.8-27B-Uncensored          # q4_K_M by default
ollama run orcarouter/Qwen3.8-27B-Uncensored:iq4_xs   # any of the 16 tags

Sixteen tags from q2_K to q8_0, each verified on the actual quantized artifact before release —
ollama.com/orcarouter.

Evaluation

Measured on this exact abliterated build (served with vLLM) vs the official Qwen/Qwen3.8-27B-FP8,
using the same scripts and settings. Refusal is judged by a rule-based opening-phrase classifier —
indicative, not an LLM-judge / publication-grade number. GGUF quants are deterministic derivations and
inherit these behaviours; lower bits trade some quality (see the file table).

Safety — harmful-prompt refusal, thinking OFF (lower = more uncensored)

Safety — harmful-prompt refusal, thinking ON (enable_thinking=true)

Over-refusal — benign prompts wrongly refused (lower = better)

Capability retention — vs the official base FP8 (same scripts)

Harmful-prompt refusal collapses from 64–99% (base) to 0–6%; benign over-refusal drops
(5.6%→0.4%); capability stays within ±1.3 pts of the base. Reasoning (enable_thinking),
multi-turn tool calling (qwen3_coder), and vision (image + OCR via mmproj) all verified working on
the GGUF build. Note: the above are full-precision/FP8 numbers; expect small additional degradation at
lower quants (most visible at Q2_K / Q3).

Hardware

• Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. VRAM/RAM ≈ the file size + KV cache + (for vision)
  the ~0.9 GB mmproj. E.g. Q4_K_M fits comfortably on a 24 GB GPU with room for context.

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B.
Abliteration and quantization do not change the underlying license obligations.