license: apache-2.0
base_model: Qwen/Qwen3.8-27B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
language:

• en
• zh
  tags:
• abliterated
• uncensored
• qwen3.8
• gguf
• q4_k_m
• quantized
• llama.cpp
• red-teaming
• reasoning
• function-calling

A UD-Q4_K_XL GGUF of orcarouter/Qwen3.8-27B-Uncensored-FP8 —
itself an abliterated (refusal-removed) block-FP8 build of Qwen/Qwen3.8-27B.
This GGUF applies SSM-aware selective quantization: the Gated DeltaNet linear-attention
gating tensors (ssm_alpha, ssm_beta, ssm_out) are pinned to Q8_0 to preserve
recurrent state precision, while embeddings and bulk transformer weights quantize at standard balanced Q4_K / Q6_K to match the official Unsloth UD-Q4_K_XL profile and footprint.

⚠️ Disclaimer — read before use

This model inherits the abliteration applied by orcarouter. Its refusal alignment has been substantially removed:

• It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse.
• It is released strictly for legitimate research — interpretability, AI-safety study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for every use and for everything it generates.
• Do not deploy to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License and all laws and regulations that apply to you.
• The uploader accepts no liability for any misuse or harm arising from this model.

By downloading or using this model you acknowledge and accept the above.

Model details

Quantization Strategy — Why UD-Q4_K_XL?

Qwen3.8-27B is a hybrid architecture: 48 of its 64 layers use Gated DeltaNet linear
attention (SSM-style), while 16 layers use full multi-head attention. The SSM layers carry
recurrent gating state through ssm_alpha and ssm_beta tensors — aggressively quantizing
these with standard 4-bit introduces compounding numerical drift across the recurrent state that
degrades generation quality more than the same quantization on standard attention weights.

The UD-Q4_K_XL scheme addresses this with a selective tensor-type override:

ssm_alpha=q8_0
ssm_beta=q8_0
ssm_out=q8_0

Audited Tensor Layout

The quantization log confirms all recurrent SSM tensors correctly reach Q8_0:

blk.N.ssm_alpha.weight  [5120, 48]   bf16 → q8_0  (0.47 MiB → 0.25 MiB)
blk.N.ssm_beta.weight   [5120, 48]   bf16 → q8_0  (0.47 MiB → 0.25 MiB)
blk.N.ssm_out.weight    [6144, 5120] bf16 → q8_0  (60.00 MiB → 31.88 MiB)

Build pipeline

Source:  orcarouter/Qwen3.8-27B-Uncensored-FP8  (7 safetensors shards, 30.9 GB, FP8)
    │
    ▼  convert_hf_to_gguf.py --outtype bf16
Intermediate: qwen38_bf16.gguf  (52115 MiB, BF16)
    │
    ▼  llama-quantize --tensor-type-file ud_q4_rules.txt \
    │                 Q4_K_M  (4 threads, OCI Ampere A1 aarch64)
    ▼
Output:  Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf  (16752 MiB / 16.36 GiB, 5.14 BPW)

• Host: OCI Ampere A1 (4× Neoverse-N1 cores, 24 GB RAM, aarch64)
• llama.cpp: built from source (Sept 2026)
• Quantization time: ~968 seconds

Usage

llama.cpp CLI

./llama-cli \
  -m Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf \
  -c 8192 \
  -n 512 \
  --temp 0.7 \
  -p "Your prompt here"

llama-server (OpenAI-compatible API)

./llama-server \
  -m Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf \
  -c 16384 \
  --host 0.0.0.0 \
  --port 8080

GPU Offloading (e.g. RTX 4090 24GB)

With its 16.36 GiB weight footprint, this model offloads completely into 24 GB VRAM:

./llama-server \
  -m Qwen3.8-27B-Uncensored-UD-Q4_K_XL.gguf \
  -ngl 99 \
  -fa \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  -c 131072

Minimum RAM requirements (CPU)

24 GB unified memory (Apple Silicon M2/M3 Pro or better) or 24 GB VRAM GPU is sufficient for high-context execution.

About the source model

orcarouter/Qwen3.8-27B-Uncensored-FP8
is an abliterated, block-FP8 quantized build of Qwen/Qwen3.8-27B
by OrcaRouter.

Abliteration follows Arditi et al. (2024) — the refusal direction r is estimated at layer 38
(round(0.6 × 64)) from harmful vs. harmless residuals, then orthogonalized out of all
residual-writing matrices (o_proj, out_proj, down_proj, embed_tokens). The MTP head is
abliterated consistently. Max residual leakage: 1.8e-2.

License

Apache 2.0 — inherited from Qwen/Qwen3.8-27B.
See LICENSE.