license: apache-2.0
base_model:
- orcarouter/Qwen3.8-27B-Uncensored
tags: - compressed-tensors
- w4a16
- auto-round
- qwen3_5
- mtp
- uncensored
- hyperqwen
pipeline_tag: image-text-to-text
Qwen3.8-27B Uncensored (orcarouter), W4A16, HyperQwen-prepared
A 4-bit weight-only quant of orcarouter/Qwen3.8-27B-Uncensored, an abliteration of the official Qwen3.8-27B. It is quantized the way dbirks/Qwen3.8-27B-W4A16-AutoRound is, so HyperQwen's prep and kernels apply and plain vLLM loads it with Marlin.
- Format: compressed-tensors
pack-quantized, int4, group 128, symmetric. - Quantizer: Intel AutoRound 0.15.0, 128 samples of NeelNanda/pile-10k at 2048 tokens, 200 iterations, seed 42.
- Kept in BF16: GatedDeltaNet
in_proj_a/in_proj_b, the vision tower, the MTP head andlm_head. - MTP head included (15 tensors), so
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'works. - Already prepared for HyperQwen: int8 lm_head, embeddings and MTP module, plus the 40k-token MTP draft head, so a server can start without running the prep step. The chat template renders the reasoning-effort line at the end of the prompt, so changing effort does not break the prefix cache.
- Size: 19.5 GB before prep. Quantized in 34 minutes on one RTX 5090.
The refusal-removal results and benchmarks are on the source model card. Credit for the abliteration goes to orcarouter, and for the base model to the Qwen team (Apache 2.0).