license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
tags:

• exl3
• exllamav3
• qwen3.8
• abliterated
• uncensored
• mtp
• quantized
  language:
• en
• fr
• zh
  pipeline_tag: text-generation

Qwen3.8-27B-Uncensored — EXL3 4.0 bpw

EXL3 quantization of
orcarouter/Qwen3.8-27B-Uncensored,
an abliterated (refusal-removed) build of Qwen/Qwen3.8-27B.

Built with ExLlamaV3 1.4.6.

16 GB on disk — fits a single 24 GB card with room for context.

The MTP head is preserved at 8 bpw and works. GGUF conversions of this
family drop the mtp.* tensors, so they have no self-drafter. TabbyAPI logs
Using main model MTP component for drafting on load.

Sibling repos

Build

python convert.py \
  -i Qwen3.8-27B-Uncensored \
  -o Qwen3.8-27B-Uncensored-exl3-4bpw \
  -w /tmp/exl3-work \
  -b 4.0 -hb 6 -mb 8 -vb 16 \
  -d 0,1,2,3 -v

Default bundled calibration corpus (wiki 50, C4 20, code 20, random tokens
20, technical 10, multilingual 10, tiny 5), 250 rows x 2048 columns.

Usage — TabbyAPI

model:
  model_dir: /path/to/models
  model_name: Qwen3.8-27B-Uncensored-exl3-4bpw
  cache_size: 32768
  cache_mode: "8,8"
  tensor_parallel: true

draft_model:
  draft_mode: mtp

Benchmarks

Hardware: 4x RTX 4000 Ada (20 GiB, sm89), PCIe, no NVLink, 2x Xeon
Gold 5318Y. AIPerf, streaming, ignore_eos, seed 42, with warmup.
ExLlamaV3 1.4.6 + TabbyAPI, tensor-parallel, cache_mode "8,8", MTP on.

Bitrate does almost nothing here

ISL 1000 / OSL 500 / concurrency 4 / 20 requests:

75% more weight from 4 to 8 bpw costs only 6% latency. On four
tensor-parallel cards over PCIe, weight bandwidth is not the bottleneck —
inter-GPU synchronization dominates. Pick the bitrate on VRAM budget and
quality, not on speed.

The 8 bpw TTFT being lowest of the three is noise; 20 requests can't resolve
5% gaps.

vs vLLM (same weights, FP8 build)

Long prompts (ISL 8000 / conc. 2): vLLM TTFT 5,508 ms vs 9,481 ms, prefill
1,733 vs 1,277 tok/s.

vLLM is faster in every served scenario on this box. EXL3 wins on VRAM and
on single-stream short prompts (63 vs ~45 tok/s in chat.py).

tensor_parallel is a trade, not a win

Helps compute-bound prefill, hurts memory-bound decode. State which one you
ran when quoting EXL3 numbers.

Caveat: PCIe, no NVLink, SymmMemCommunicator unavailable on sm89. TP
results are likely better on NVLink systems. Treat these as a floor.

Quality

Not benchmarked at 4.0 bpw. For reference on the same abliterated weights,
the FP8 build scores 88.0% on MMLU-Pro (business subset, 100 questions, 2
unparsed) vs 89.0% for official Qwen3.8-27B-FP8 — within noise. How 4.0 bpw
compares has not been verified. At this bitrate expect measurable
degradation vs the 6 and 8 bpw siblings.

Safety

Inherited from the base model: safety alignment has been substantially
removed via abliteration. It will comply with requests the original
Qwen3.8-27B refuses, and has no meaningful built-in guardrails. Released for
research and controlled experimentation. Add your own moderation layer
before any deployment. You are responsible for what you do with it.

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B.

Credits

• Qwen — base model
• orcarouter — abliterated build
• turboderp — ExLlamaV3 / EXL3
• theroyallab — TabbyAPI