license: apache-2.0
base_model:
- orcarouter/Qwen3.8-27B-Uncensored
tags: - compressed-tensors
- w4a16
- auto-round
- qwen3_5
- mtp
- uncensored
- hyperqwen
pipeline_tag: image-text-to-text
Qwen3.8-27B Uncensored (orcarouter), W4A16 compressed-tensors
A 4-bit weight-only quant of orcarouter/Qwen3.8-27B-Uncensored, an abliteration of the official Qwen3.8-27B. It is quantized the way dbirks/Qwen3.8-27B-W4A16-AutoRound is, so HyperQwen's prep and kernels apply and plain vLLM loads it with Marlin.
- Format: compressed-tensors
pack-quantized, int4, group 128, symmetric. - Quantizer: Intel AutoRound 0.15.0, 128 samples of NeelNanda/pile-10k at 2048 tokens, 200 iterations, seed 42.
- Kept in BF16: GatedDeltaNet
in_proj_a/in_proj_b, the vision tower, the MTP head andlm_head. - MTP head included (15 tensors), so
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'works. - Size: 19.5 GB. Quantized in 34 minutes on one RTX 5090.
The refusal-removal results and benchmarks are on the source model card. Credit for the abliteration goes to orcarouter, and for the base model to the Qwen team (Apache 2.0).