license: apache-2.0
base_model: noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
language:
- en
- zh
tags: - qwen3.8
- qwen3_5
- uncensored
- abliterated
- compressed-tensors
- autoround
- gptq
- w4a16
- int4
- mtp
- speculative-decoding
- vllm
- rtx-3090
Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4
A standalone, serving-optimized derivative ofnoon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound,
built and measured for low-latency MTP speculative decoding on one 24 GB RTX
3090-class GPU usingsyv-ai/qwen38-27b-rtx3090.
This repository contains the complete checkpoint. It does not require the
upstream weight repository after download.
[!WARNING]
This checkpoint inherits substantial safety-alignment removal from its
abliterated upstream model. It can comply with harmful, unethical,
offensive, or illegal requests and should not be treated as having meaningful
built-in guardrails. It is published for legitimate local use, evaluation,
interpretability, red-teaming, and safety research. Do not expose it to
untrusted users without an independent safety layer. Users are responsible
for how they deploy and use it.
Provenance
The model lineage is:
Qwen/Qwen3.8-27B— original Apache-2.0 model.orcarouter/Qwen3.8-27B-Uncensored— abliterated derivative.noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound— AutoRound W4A16 quantization.- This checkpoint — further quantization and MTP serving optimization.
The immediate source revision was0e10c9f6b5b8a97fba199e82c49690d272f776ce. The original abliteration and
the initial W4A16 quantization were performed by the respective upstream
authors, not by this repository's uploader.
What changed
- Transformer body: retained from the source checkpoint as symmetric W4A16,
group size 128. - Token embeddings: symmetric INT8, group size 128.
lm_head: calibrated symmetric GPTQ INT4, group size 128.- Native MTP module: calibrated symmetric GPTQ INT4, group size 128.
- Draft head: 40,960 rows sliced from the calibrated INT4
lm_head. - Draft vocabulary: selected from this checkpoint's own generated output-token
distribution instead of being inherited from the aligned Qwen checkpoint. - Vision tower and multimodal processor files are retained.
The calibration run generated 6,761 responses containing 5,500,353 output
tokens. The resulting draft vocabulary covered 97.77% of a held-out set of
571,632 output tokens. Hidden-state calibration used 1,200 sequences and
1,273,013 rows. The lm_head INT4 calibration measured KL 0.00308 with GPTQ,
compared with 0.00695 for round-to-nearest.
Measured RTX 3090 Ti profile
Measurements below were made locally on an NVIDIA GeForce RTX 3090 Ti (24 GB)
with the patched vLLM 0.27.1 runtime from syv-ai/qwen38-27b-rtx3090:
| Measurement | Result |
|---|---|
| Serving profile | MTP-4, BF16 KV, FlashAttention 2, prefix cache |
| Maximum configured model length | 80,000 tokens |
| Startup KV capacity | 80,392 tokens |
| Fixed 512-token non-thinking decode | 131.5 tok/s mean |
| Live MTP mean acceptance length | 3.03 tokens/step |
| Per-position MTP acceptance | 82.0%, 58.5%, 40.7%, 21.9% |
| GSM8K guardrail | 29/30 (96.7%) |
| Cold long-context retrieval | exact needle at a 69,993-token prompt |
The 512-token decode measurements were 132.4, 131.4, and 130.7 tok/s. These
are single-machine serving measurements rather than hardware-independent model
benchmarks; other prompts, sampling settings, drivers, runtimes, and GPUs will
produce different results.
Recommended serving setup
The measured MTP performance requires the patched runtime and launch profile insyv-ai/qwen38-27b-rtx3090.
The checkpoint's MTP and reduced draft vocabulary are not a promise of identical
performance with an unpatched stock vLLM installation.
git clone https://github.com/syv-ai/qwen38-27b-rtx3090.git
cd qwen38-27b-rtx3090
hf download protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \
--local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4
printf '%s\n' \
'MODEL=/app/models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4' \
'CTX=fast' \
'SPEC=mtp' \
'PREFIX_CACHE=1' \
'GPU_UTIL=0.95' \
'MAX_LEN=80000' \
'MAX_SEQS=4' > .env
docker compose --profile single up -d --no-deps single
The first launch can take several minutes while kernels compile and CUDA graphs
are captured. Reduce GPU_UTIL, MAX_LEN, or both if the GPU is also driving a
desktop or if another process needs VRAM. The exact 80,000-token profile was
validated on the uploader's otherwise dedicated 3090 Ti; a display workload can
reduce the available margin.
Validation notes
- Core chat, reasoning, tool/JSON, streaming, logprobs, penalties, and long-prompt
API tests passed locally. - The long-context test retrieved the target text exactly at 69,993 prompt
tokens in 70.73 seconds from a cold request. - Seeded probabilistic speculative decoding was not byte-deterministic in the
local smoke suite. - The model has no independent safety layer. The absence of refusal must never
be interpreted as correctness, legality, or safety.
License and attribution
Released under Apache License 2.0, inherited from the Qwen and intermediate
checkpoints. See LICENSE. This model card identifies the upstream
works and gives a prominent description of the modifications made to the
checkpoint.