license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• abliterated
• uncensored
• qwen
• qwen3
• qwen3.8
• 4-bit
• w4a16
• int8
• compressed-tensors
• vllm
• marlin
• mtp
• reasoning
• function-calling
• vision-language
• ampere
• rtx-3090

The abliterated (refusal-removed) Qwen3.8-27B at ~17 GB — fits one 24 GB card with room
for KV cache, keeps the full vision tower, the MTP head, tool calling, and the flexible
thinking control of the base. Weights-only quantization: the abliteration edit from the source
release is carried through byte-for-byte.

Lineage

Qwen/Qwen3.8-27B                       (upstream, Apache 2.0)
  └─ orcarouter/Qwen3.8-27B-Uncensored (BF16 abliterated finetune — refusal direction removed)
       └─ Qwen3.8-27B-Uncensored-BF16  (56 GB, 18 shards — local quantization source)
            └─ THIS MODEL              (W4A16 g128 + int8 embed/lm_head/MTP, 2026-09-10)

Quantized from the local BF16 source tree G:\models\Qwen3.8-27B-Uncensored-BF16 (byte-identical
to orcarouter/Qwen3.8-27B-Uncensored,
revision 404ea47aaa5d8a8b00049c9e9750089aca011ab2 — that source repo is gated with auto-approval on
HF; accept the agreement to download it).

What changed vs the BF16 source

The template translation is deliberate: the upstream template raises a TemplateError (vLLM returns
HTTP 400) when a client sends reasoning_effort: "minimal" / "high" / "max". Everything else in
the template is byte-identical to upstream. A .bak-effort copy of the original was kept locally and
is not part of this release.

Quantization recipe

• Tool: Intel AutoRound 0.16.0.dev148+g1139c323 (with an upstreamed pack guard for unquantized
  linear_attn/visual layers), scheme W4A16, --bits 4 --group_size 128, --nsamples 128 --seqlen 2048, --batch_size 4, format auto_round:llm_compressor.
• Hardware: one RTX 3090 (24 GB) + one RTX 3080 Ti (12 GB), block-streaming calibration.
• Post-quant (repo pipeline, syv-ai/qwen38-27b-rtx3090):
  int8 lm_head / embed_tokens / MTP-MLP packing, 40k draft-vocab build, verify.sh gate.
• 411 tensors quantized to packed int4 (+ F16 scales); 790 tensors left BF16 (norms, vision, MTP
  embeddings); full tensor manifest via model.safetensors.index.json.

Quality on this exact artifact

Measured serving this checkpoint with vLLM 0.28.0 + DFlash2 speculative decoding (k=7) on an RTX
3090:

The capability-retention evals in the source card (MMLU / MMLU-Pro / CMMLU / safety suites) were
measured on the BF16 source and its FP8/GGUF derivatives — quantization is the only change here; the
4-bit weights inherit those numbers up to the usual W4A16 cost, which GSM8K puts at none measurable
(98.0 with thinking on vs 95.5 without on the same stack).

Daily-driver verified: Marlin W4A16 serving, multi-turn tool calling (qwen3_coder parser),
thinking control, DFlash2 speculative decoding, loopback OpenAI-compatible API.

Serving (vLLM ≥ 0.28.0)

vllm serve /path/to/Qwen3.8-27B-Uncensored-W4A16-g128 \
  --served-model-name qwen3.8-27b \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

• RTX 3090 (24 GB): weights ~15.8 GiB on load; --gpu-memory-utilization 0.93 leaves a large KV
  pool for 128k context. On WSL2/Windows, 0.93 (not 0.97) avoids a startup OOM.
• Native context is 262,144; the official 1M recipe (static YaRN via --hf-overrides) applies.
• Speculative decoding — pair with the DFlash2 drafter per the official vLLM recipe:
  --speculative-config '{"method":"dflash","model":"incoai/Qwen3.8-27B-DFlash2","num_speculative_tokens":7}'
  (mtp_draft_vocab_ids.pt in this repo serves the same draft-vocab pipeline).
• --reasoning-parser qwen3 is effectively required: the template opens every assistant turn with
  <arg_key>, so without it the reasoning lands in content and eats the token budget.
• FP8/NVFP4 are not options here (Ampere has no FP8/NVFP4 tensor cores) — that is what this
  release is for.

Chat Completions / thinking control

Thinking is on by default. Toggle per request:

resp = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=messages,
    reasoning_effort="low",            # low | medium (default) | xhigh — plus OpenAI aliases minimal/high/max
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},  # direct answer, no <arg_key>
)

preserve_thinking defaults to true (prior-turn reasoning retained). Recommended sampling
(generation_config.json ships these): thinking — temperature=1.0, top_p=0.95, top_k=20;
non-thinking — temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5.

Files

Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the
refusal direction out of the residual stream), inherited unchanged from the BF16 source.

• It will comply with harmful, unethical, offensive, or illegal requests that the original
  Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism
  study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do
  not deploy it to end users or in production without adding your own safety, moderation, and
  abuse-prevention layers.
• Use must comply with the Apache 2.0 License inherited
  from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm arising from this model. Its
  outputs do not reflect the views of the uploaders, of OrcaRouter, or of Qwen / Alibaba.

By downloading or using this model you acknowledge and accept the above.

Intended use / out of scope

• Intended: research into refusal mechanisms, alignment, interpretability; red-teaming and safety
  evaluation in controlled environments; single-user local inference on 24 GB Ampere hardware.
• Out of scope: any use violating the base model's Apache 2.0 license or applicable law; deployment
  to the public or end users without additional safety and moderation layers; content intended to harm.

Bias, risks, and limitations

• Safety guardrails removed — the model will produce harmful, biased, or offensive content on request.
• Inherits biases/limitations of the base Qwen3.8-27B and of the abliteration edit (see the
  BF16 source card for the full evaluation).
• 4-bit weights: fine-tuning on top of this checkpoint should start from the BF16 source instead.

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B through the OrcaRouter abliterated release.
Quantization does not change the underlying license obligations.

Acknowledgements

• Qwen/Qwen3.8-27B — base model (Apache 2.0)
• orcarouter/Qwen3.8-27B-Uncensored —
  abliterated BF16 source (OrcaRouter / Continuum-AI-Corp)
• incoai/Qwen3.8-27B-DFlash2 — DFlash2 drafter
• Tooling: syv-ai/qwen38-27b-rtx3090 (patch series,
  post-quant pipeline), Intel AutoRound