language:
- en
- zh
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
base_model: orcarouter/Qwen3.8-27B-Uncensored-FP8
base_model_relation: quantized
quantized_by: vcruz305
tags: - gguf
- qwen
- qwen3.8
- llama.cpp
- uncensored
- abliterated
Qwen3.8-27B Uncensored GGUF
Standalone llama.cpp K-quants of orcarouter/Qwen3.8-27B-Uncensored-FP8, a community abliterated block-FP8 of Qwen/Qwen3.8-27B.
This is not official Qwen. It is also not vcruz305/Qwen3.8-27B-GGUF — that pack is the official BF16 trunk.
What is in these files
27B dense hybrid-attention (qwen35). 64 language-trunk blocks (0–63). Hidden 5120, FFN 17408. Native context 262,144.
MTP / nextn is omitted (--no-mtp). Speculative decode does not make the model smarter; the extra head steals KV on 12–24 GB cards. Need vision? Pair a separate mmproj. Need MTP? Use another pack.
The source checkpoint had the refusal direction removed (abliteration). These GGUFs inherit that behavior.
Chat template
Official 3.8 jinja wraps every assistant turn in <think>…</think> even when reasoning is empty, then opens another <think> on generate. That truncates multi-turn agents.
These GGUFs bake a fixed template. Use --jinja. A standalone chat_template.jinja ships in the repo if an older copy is still on disk.
llama-server -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf --jinja --reasoning-format deepseek
Files
One file per quant. Byte / GiB filled in when the ladder lands.
| File | Quant | Bytes | GiB | Notes |
|---|---|---|---|---|
Qwen3.8-27B-Uncensored-Q2_K.gguf |
Q2_K | TBD | TBD | 12GB start |
Qwen3.8-27B-Uncensored-Q3_K_M.gguf |
Q3_K_M | TBD | TBD | 16GB |
Qwen3.8-27B-Uncensored-Q4_K_M.gguf |
Q4_K_M | TBD | TBD | 24GB start — default |
Qwen3.8-27B-Uncensored-Q5_K_M.gguf |
Q5_K_M | TBD | TBD | 24GB comfortable |
Qwen3.8-27B-Uncensored-Q6_K.gguf |
Q6_K | TBD | TBD | Largest full-GPU on 24GB Turing |
Qwen3.8-27B-Uncensored-Q8_0.gguf |
Q8_0 | TBD | TBD | 32GB+; will not -ngl 99 on 24GB |
Download
Use hf_xet. Do not git clone.
export HF_XET_HIGH_PERFORMANCE=1
hf download vcruz305/Qwen3.8-27B-Uncensored-GGUF \
--local-dir Qwen3.8-27B-Uncensored-GGUF \
--include "Qwen3.8-27B-Uncensored-Q4_K_M.gguf"
Change --include for the quant you want.
How to run
Needs llama.cpp new enough for qwen35 (Gated DeltaNet hybrid).
24GB (default Q4_K_M):
llama-server \
-m Qwen3.8-27B-Uncensored-GGUF/Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
-a qwen38-27b-unc \
--host 127.0.0.1 --port 8085 \
-ngl 99 -c 32768 -np 1 --jinja --reasoning-format deepseek
Q6_K is the largest file that still full-offloads 24GB Turing. Q8_0 does not (-ngl 99 will not fit).
Intended use
Local llama.cpp serving of the uncensored 27B trunk: research, red-team, and unfiltered generation in a setting you control.
Out of scope: treating this as official Qwen or as a drop-in for vcruz305/Qwen3.8-27B-GGUF; deploying to end users without your own filters; any use that breaks Apache-2.0 or the law.
Bias, risks, limitations
Safety alignment was removed at the source. The model will answer requests the official 27B would refuse. It still carries the bias and failure modes of Qwen3.8-27B, plus K-quant error. These files are language-only (no vision tower, no MTP).
Source
- Uncensored FP8: https://huggingface.co/orcarouter/Qwen3.8-27B-Uncensored-FP8
- Official base: https://huggingface.co/Qwen/Qwen3.8-27B
- Official-trunk GGUFs (different weights): https://huggingface.co/vcruz305/Qwen3.8-27B-GGUF
- Convert:
convert_hf_to_gguf.py --outtype f16 --no-mtp→llama-quantizeK-quants - License: Apache-2.0, same as Qwen3.8-27B and the orcarouter card
Credits
Abliteration and FP8: orcarouter. Base model: Qwen / Alibaba. GGUF pack: Victor Cruz (vcruz305).