license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• qwen
• qwen3.8
• nvfp4
• fp8
• fp8-kv
• uncensored
• abliterated
• vision-language
• function-calling
• modelopt

Qwen3.8-27B-NVFP4-Uncensored

This is a ModelOpt mixed-precision derivative of
orcarouter/Qwen3.8-27B-Uncensored,
generated from source revision
9878936be9458522b5aeed0e13476bb8426f57f0.

This is not a pure all-NVFP4 checkpoint. Its weight path combines NVFP4,
FP8, and retained higher-precision tensors; the exact composition is described
below. The repository is intended to be a reproducible vLLM/ModelOpt artifact,
not a claim that every Transformers backend can execute the quantized layers
without matching ModelOpt support.

The model preserves the Qwen3.8 vision-language tower and MTP head. It is an
abliterated/uncensored model with substantially reduced built-in refusal
behavior. It is intended for controlled research, evaluation, and local
experimentation. Add your own moderation and access controls before any
user-facing deployment.

Quantization

• MLP and language-model-head weights: W4A16_NVFP4, group size 16. The
  exported metadata identifies 193 such target layers.
• Attention and linear-attention projections: ModelOpt FP8 mixed precision,
  covering 208 target layers.
• KV cache: FP8 E4M3 with 32 exported scalar scale tensors (16 K and 16 V)
  from the data-calibrated ModelOpt kv_fp8 recipe. The audited scales are
  finite, positive, and non-unit.
• Vision, MTP, and hybrid-state tensor families are retained in the unified
  checkpoint and excluded from the weight-quantization target map.
• Export format: unified Hugging Face safetensors checkpoint.

The full export contains 2,033 indexed tensors across three safetensors
shards. ModelOpt metadata records MIXED_PRECISION weights and
kv_cache_quant_algo: FP8, produced with ModelOpt
0.47.0.dev81+ga2fbac7ba.

Calibration

The KV scales were calibrated with 256 pre-rendered, text-only examples at a
2,048-token calibration sequence length, batch size 1, and
enable_thinking=false. The corpus was generic rather than application-owned:
128 general instruction/chat rows, 64 code/technical rows, 32 structured-output
rows, and 32 longer-context rows. No images or videos were used for calibration.
This establishes a scale-aware FP8-KV export for the tested distribution; it is
not a universal multimodal or application-specific calibration claim.

vLLM serving

This artifact was validated with vllm/vllm-openai:v0.27.1. The relevant
starting flags are:

--quantization modelopt_fp4
--kv-cache-dtype fp8_e4m3
--max-model-len 262144
--trust-remote-code

On the validation stack, the requested modelopt_fp4 flag resolved to vLLM's
modelopt_mixed path. The resolved KV dtype was float8_e4m3fn.

For the tested Qwen XML tool-calling route, also use:

--enable-auto-tool-choice --tool-call-parser qwen3_xml
--default-chat-template-kwargs '{"enable_thinking":false}'

Validation notes

In the project's Experiment 014, the candidate returned 56/56 HTTP 200
responses with complete streaming [DONE] markers, zero stream parse errors,
and zero reasoning leaks. The explicit response-contract path passed 8/8.
The unchanged no-contract baseline passed 12/24 because the known
fenced-JSON and HH:MM formatting behaviors remained; these were output
contract misses, not cache-load failures. A short long-context load returned
24/24 complete streams at concurrency levels 1, 2, and 4; its diagnostic
fixture oracle passed 21/24. vLLM reported an allocator capacity of 2,491,134
FP8-KV tokens while the configured maximum sequence length was 262,144. The
allocator figure is a host/runtime capacity diagnostic, not a claim that the
model supports a 2.49-million-token context.

These are narrow, single-host exploratory results, not a general quality or
production-readiness claim.

Important caveats

• On the project's NVIDIA GB10, vLLM uses the Marlin software-FP4 path because
  the GPU has no native FP4 computation support; compute-heavy performance may
  differ from a native-FP4 GPU.
• The checkpoint does not include separate q-scale metadata. In the tested
  vLLM FP8 attention path, q scaling therefore falls back to the K scale;
  q/probability scales were not independently calibrated.
• The calibration and validation were text-focused. Multimodal quality,
  application-wide quality, independent BF16-vs-FP8 throughput, and production
  readiness remain unproven. The tested persistent WebUI route continued to
  use BF16 KV as its correctness-first default.
• The model's uncensored behavior means it may produce harmful or illegal
  content. Follow the Apache 2.0 license, applicable law, and your own safety
  requirements.

License and provenance

The Apache 2.0 license file from the derived checkpoint is included in this
repository. Users are responsible for complying with the base model's terms,
the derivative model's terms, and all applicable laws.