license: apache-2.0
base_model: Qwen/Qwen3.8-27B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
language:
- en
- zh
tags: - abliterated
- uncensored
- qwen3.8
- gguf
- q4_k_m
- quantized
- llama.cpp
- red-teaming
- reasoning
- function-calling
Qwen3.8-27B-Uncensored-GGUF
UD-Q4_K_XL GGUF quantization of orcarouter/Qwen3.8-27B-Uncensored-FP8 — SSM-aware selective quantization for the Gated DeltaNet hybrid
A UD-Q4_K_XL GGUF of
orcarouter/Qwen3.8-27B-Uncensored-FP8—
itself an abliterated (refusal-removed) block-FP8 build ofQwen/Qwen3.8-27B.
This GGUF applies SSM-aware selective quantization: the Gated DeltaNet linear-attention
gating tensors (ssm_alpha,ssm_beta,ssm_out) are pinned to Q8_0 to preserve
recurrent state precision, while embeddings and bulk transformer weights quantize at standard balanced Q4_K / Q6_K to match the official Unsloth UD-Q4_K_XL profile and footprint.
⚠️ Disclaimer — read before use
This model inherits the abliteration applied by orcarouter. Its refusal alignment has been substantially removed:
- It will comply with harmful, unethical, offensive, or illegal requests that the original
Qwen3.8-27Bwould refuse. - It is released strictly for legitimate research — interpretability, AI-safety study, red-teaming, robustness evaluation, and controlled experiments.
- You assume full responsibility and liability for every use and for everything it generates.
- Do not deploy to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
- Use must comply with the Apache 2.0 License and all laws and regulations that apply to you.
- The uploader accepts no liability for any misuse or harm arising from this model.
By downloading or using this model you acknowledge and accept the above.
Model details
| Base model | Qwen/Qwen3.8-27B |
| Source FP8 | orcarouter/Qwen3.8-27B-Uncensored-FP8 |
| Architecture | Qwen3_5ForConditionalGeneration — 64 layers, hidden 5120, hybrid Gated DeltaNet (48 linear-attention + 16 full-attention, interval 4), MTP speculative-decoding head |
| Modification | Abliteration (refusal-direction removal) by orcarouter; converted FP8→BF16→GGUF |
| Quantization | UD-Q4_K_XL — Standard Q4_K_M base + recurrent SSM tensors (ssm_alpha, ssm_beta, ssm_out) pinned to Q8_0 |
| File | Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf — single file, 16.36 GiB |
| BPW | 5.14 bits per weight (52,115 MiB BF16 → 16,752 MiB) |
| Context | 262,144 tokens |
| Tooling | Built with llama.cpp convert_hf_to_gguf.py + llama-quantize |
Quantization Strategy — Why UD-Q4_K_XL?
Qwen3.8-27B is a hybrid architecture: 48 of its 64 layers use Gated DeltaNet linear
attention (SSM-style), while 16 layers use full multi-head attention. The SSM layers carry
recurrent gating state through ssm_alpha and ssm_beta tensors — aggressively quantizing
these with standard 4-bit introduces compounding numerical drift across the recurrent state that
degrades generation quality more than the same quantization on standard attention weights.
The UD-Q4_K_XL scheme addresses this with a selective tensor-type override:
ssm_alpha=q8_0
ssm_beta=q8_0
ssm_out=q8_0
Audited Tensor Layout
| Tensor group | Quant type | Rationale |
|---|---|---|
ssm_alpha, ssm_beta (gating) |
Q8_0 | Recurrent gating — precision-critical |
ssm_out (projection) |
Q8_0 | Output of linear attention path |
ssm_conv1d, ssm_a, ssm_dt.bias |
f32 (unchanged) | Already small / scalar tensors |
token_embd (embeddings) |
Q4_K | Standard balanced token embedding |
output (LM head) |
Q6_K | Higher precision output projection |
attn_output, ffn_down (every layer) |
Q4_K / Q6_K | Standard transformer attention & feedforward paths |
Bulk weights (attn_*, ffn_*) |
Q4_K_M | Majority of parameters |
The quantization log confirms all recurrent SSM tensors correctly reach Q8_0:
blk.N.ssm_alpha.weight [5120, 48] bf16 → q8_0 (0.47 MiB → 0.25 MiB)
blk.N.ssm_beta.weight [5120, 48] bf16 → q8_0 (0.47 MiB → 0.25 MiB)
blk.N.ssm_out.weight [6144, 5120] bf16 → q8_0 (60.00 MiB → 31.88 MiB)
Build pipeline
Source: orcarouter/Qwen3.8-27B-Uncensored-FP8 (7 safetensors shards, 30.9 GB, FP8)
│
▼ convert_hf_to_gguf.py --outtype bf16
Intermediate: qwen38_bf16.gguf (52115 MiB, BF16)
│
▼ llama-quantize --tensor-type-file ud_q4_rules.txt \
│ Q4_K_M (4 threads, OCI Ampere A1 aarch64)
▼
Output: Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf (16752 MiB / 16.36 GiB, 5.14 BPW)
- Host: OCI Ampere A1 (4× Neoverse-N1 cores, 24 GB RAM, aarch64)
- llama.cpp: built from source (Sept 2026)
- Quantization time: ~968 seconds
Usage
llama.cpp CLI
./llama-cli \
-m Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf \
-c 8192 \
-n 512 \
--temp 0.7 \
-p "Your prompt here"
llama-server (OpenAI-compatible API)
./llama-server \
-m Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf \
-c 16384 \
--host 0.0.0.0 \
--port 8080
GPU Offloading (e.g. RTX 4090 24GB)
With its 16.36 GiB weight footprint, this model offloads completely into 24 GB VRAM:
./llama-server \
-m Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf \
-ngl 99 \
-fa \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
-c 131072
Minimum RAM requirements (CPU)
| Context | Estimated RAM |
|---|---|
| 2K tokens | ~19 GB |
| 8K tokens | ~21 GB |
| 32K tokens | ~26 GB |
24 GB unified memory (Apple Silicon M2/M3 Pro or better) or 24 GB VRAM GPU is sufficient for high-context execution.
About the source model
orcarouter/Qwen3.8-27B-Uncensored-FP8
is an abliterated, block-FP8 quantized build of Qwen/Qwen3.8-27B
by OrcaRouter.
Abliteration follows Arditi et al. (2024) — the refusal direction r is estimated at layer 38
(round(0.6 × 64)) from harmful vs. harmless residuals, then orthogonalized out of all
residual-writing matrices (o_proj, out_proj, down_proj, embed_tokens). The MTP head is
abliterated consistently. Max residual leakage: 1.8e-2.
License
Apache 2.0 — inherited from Qwen/Qwen3.8-27B.
See LICENSE.