license: apache-2.0
base_model: noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
language:

• en
• zh
  tags:
• qwen3.8
• qwen3_5
• uncensored
• abliterated
• compressed-tensors
• autoround
• gptq
• w4a16
• int4
• mtp
• dflash2
• speculative-decoding
• vllm
• vllm-0.28.0
• rtx-3090

Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4

A standalone, serving-optimized derivative of
noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound,
built and measured for low-latency DFlash2 and MTP speculative decoding on one
24 GB RTX 3090-class GPU using
syv-ai/qwen38-27b-rtx3090.

This repository contains the complete checkpoint. It does not require the
upstream weight repository after download.

[!WARNING]
This checkpoint inherits substantial safety-alignment removal from its
abliterated upstream model. It can comply with harmful, unethical,
offensive, or illegal requests and should not be treated as having meaningful
built-in guardrails. It is published for legitimate local use, evaluation,
interpretability, red-teaming, and safety research. Do not expose it to
untrusted users without an independent safety layer. Users are responsible
for how they deploy and use it.

2026-09-13 DFlash2 / vLLM 0.28.0 update

The checkpoint weights are unchanged. A controlled A/B on an RTX 3090 found
that the native DFlash2 runner in patched vLLM 0.28.0 materially improves
single-request decoding over MTP4. DFlash2 uses the external quantized drafter
syvai/Qwen3.8-27B-DFlash2-W4A16.

Against vLLM 0.28.0 + MTP4, DFlash2 improved sampled median decode by 34.2%,
emitted tokens/step by 0.951, and paired median speed by 49.2%; it was faster on
93/96 requests. All eight prompt categories improved. A 55,050-token retrieval
test returned the exact target. Full prompts, raw results, boot logs, scripts,
method notes, and GPU3 promotion evidence are in
evaluation/dflash2-vllm-0.28.0/.

Recommended profile:

SPEC=dflash2
DFLASH_TOKENS=7
DRAFT_SAMPLE=probabilistic
CTX=fast
PREFIX_CACHE=1
GPU_UTIL=0.93
MAX_LEN=65536
MAX_SEQS=8

Use DRAFT_SAMPLE=probabilistic explicitly. Long greedy generations are not
guaranteed byte-identical between speculative modes, although no malformed or
collapsed output was observed. vLLM 0.28.0 also accepts
thinking_token_budget where the earlier smoke test expected HTTP 400.

Provenance

The model lineage is:

1. Qwen/Qwen3.8-27B — original Apache-2.0 model.
2. orcarouter/Qwen3.8-27B-Uncensored — abliterated derivative.
3. noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound — AutoRound W4A16 quantization.
4. This checkpoint — further quantization and MTP serving optimization.

The immediate source revision was
0e10c9f6b5b8a97fba199e82c49690d272f776ce. The original abliteration and
the initial W4A16 quantization were performed by the respective upstream
authors, not by this repository's uploader.

What changed

• Transformer body: retained from the source checkpoint as symmetric W4A16,
  group size 128.
• Token embeddings: symmetric INT8, group size 128.
• lm_head: calibrated symmetric GPTQ INT4, group size 128.
• Native MTP module: calibrated symmetric GPTQ INT4, group size 128.
• Draft head: 40,960 rows sliced from the calibrated INT4 lm_head.
• Draft vocabulary: selected from this checkpoint's own generated output-token
  distribution instead of being inherited from the aligned Qwen checkpoint.
• Vision tower and multimodal processor files are retained.

The calibration run generated 6,761 responses containing 5,500,353 output
tokens. The resulting draft vocabulary covered 97.77% of a held-out set of
571,632 output tokens. Hidden-state calibration used 1,200 sequences and
1,273,013 rows. The lm_head INT4 calibration measured KL 0.00308 with GPTQ,
compared with 0.00695 for round-to-nearest.

Evaluation evidence

The linked MP4 is a 1.0× replay of a real streamed request using its recorded
SSE chunk timestamps, with synchronized GPU telemetry. It is not a fabricated
typing animation. The raw trace, evaluator-native results, scripts, exact
version pins, and scoring notes are published in
evaluation/.

Measured RTX 3090 Ti profile

Measurements below were made locally on an NVIDIA GeForce RTX 3090 Ti (24 GB)
with the patched vLLM 0.27.1 runtime from syv-ai/qwen38-27b-rtx3090:

Decode rate is measured from the first non-empty streamed content chunk to the
last, excluding time-to-first-token; every request produced exactly 512 output
tokens. Aggregate throughput includes cohort wall time. These are
single-machine serving measurements rather than hardware-independent model
benchmarks; prompt, sampling, driver, runtime, clocks, thermals, and GPU model
all affect the result.

Quality evaluation

The MMLU-Pro result matches the immediate upstream W4A16 model card's reported
76.1% to the displayed precision. This is encouraging evidence that the
serving-specific head calibration did not create a measurable additional loss
on this benchmark, but it is not a controlled ablation of quantization and
abliteration effects.

The current pinned IFBench revision contains verifier fixes made after the much
higher historical result in the runtime repository. Those values are not
treated as directly comparable. The lower current score is published here
deliberately because this checkpoint still needs broader independent testing.

MMMU is a clear weakness. Its 24.56% result is near the chance region, and
891/900 responses exhausted the official 128-token generation cap instead of
following the task's direct-answer instruction. The retained vision tower makes
multimodal input functional, but this evidence does not establish strong vision
quality for this derivative.

Recommended serving setup

The measured speculative-decoding performance requires the patched runtime and launch profile in
syv-ai/qwen38-27b-rtx3090.
The results above use commit
1834917
and immutable image ghcr.io/syv-ai/qwen38-27b-rtx3090:sha-1834917.
They are not a promise of identical performance with stock vLLM.

git clone https://github.com/syv-ai/qwen38-27b-rtx3090.git
cd qwen38-27b-rtx3090
git checkout 18349177b7962ef1d699dc154844f9c04317a474

hf download Slaven71/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \
  --local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4

hf download syvai/Qwen3.8-27B-DFlash2-W4A16 \
  --local-dir models/Qwen3.8-27B-DFlash2-W4A16

printf '%s\n' \
  'MODEL=/app/models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4' \
  'CTX=fast' \
  'SPEC=dflash2' \
  'DFLASH_TOKENS=7' \
  'DRAFT_SAMPLE=probabilistic' \
  'PREFIX_CACHE=1' \
  'GPU_UTIL=0.93' \
  'MAX_LEN=65536' \
  'MAX_SEQS=8' > .env

docker compose --profile single up -d --no-deps single

The first launch can take several minutes while kernels compile and CUDA graphs
are captured. Reduce GPU_UTIL, MAX_LEN, or both if the GPU also drives a
desktop or another process needs VRAM. Use SPEC=mtp for the older
self-contained MTP4 profile.

Validation notes

• Core chat, reasoning, tool/JSON, streaming, logprobs, penalties, vision, and
  long-prompt API tests passed locally.
• The long-context suite placed distinct targets at three depths in prompts of
  approximately 8k, 32k, 64k, and 76k tokens. All 12 were retrieved exactly.
• A paired 50-prompt IFBench study showed a practical cost to reasoning in this
  serving profile: with a 2,048-token completion cap, thinking-on exhausted the
  budget before producing a final answer on 34/50 prompts. Exact-format tasks
  should normally use thinking off, while reasoning tasks need a larger budget.
• MTP acceptance is workload-dependent. Speculative verification preserves the
  target distribution; acceptance changes speed, not model intelligence.
• The current patched runtime's optimized multimodal path was not stable enough
  for a 900-item benchmark: full-run attempts using MTP and target-only CUDA
  graphs stopped making progress after roughly 50 completed requests, and MTP
  dry runs varied with scheduling. MMMU therefore measures the exact target
  checkpoint with a conservative eager target-only profile. Text MTP serving
  and the dedicated MTP speed tests remain unaffected.
• Seeded probabilistic speculative decoding was not byte-deterministic in the
  local smoke suite.
• The model has no independent safety layer. The absence of refusal must never
  be interpreted as correctness, legality, or safety.

License and attribution

Released under Apache License 2.0, inherited from the Qwen and intermediate
checkpoints. See LICENSE. This model card identifies the upstream
works and gives a prominent description of the modifications made to the
checkpoint.