license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:
- en
- zh
tags: - qwen3.8
- uncensored
- modelopt
- nvfp4
- blackwell
- rtx-5090
- sglang
- speculative-decoding
- mtp
- dspark
Qwen3.8-27B Uncensored NVFP4 RTX 5090
RTX 5090 optimized ModelOpt NVFP4 conversion of orcarouter/Qwen3.8-27B-Uncensored, produced directly from the original BF16 safetensors.
This checkpoint preserves the Qwen3.8 vision-language architecture and native MTP tensors. It is paired with fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090.
Combined release
| Component | Disk size | GPU weight allocation |
|---|---|---|
| NVFP4 target | 20.59 GB | 18.80 GB |
| BF16 DSpark drafter | 3.71 GB | 3.64 GB |
| Combined pair | 24.31 GB | 22.44 GB |
The complete 16K serving profile uses approximately 29.39 GB of GPU memory after target loading, draft loading, cache allocation, eager buffers, and CUDA graph capture. It leaves 3.22 GB free on the RTX 5090.
| Context field | Tokens |
|---|---|
| Architecture maximum | 262,144 |
| RTX 5090 launch context | 65,536 |
| Optimized active token pool | 16,384 |
Measured performance
One RTX 5090, concurrency 1, temperature 0, deterministic inference, FP8 E4M3 KV cache, and the same local prompt suite.
| Profile | Median decode | Paired speedup |
|---|---|---|
| Standard decoding | 50.74 tokens per second | 1.00 times |
| Native MTP depth 5 | 98.22 tokens per second | 1.97 times |
| Target-matched DSpark | 139.32 tokens per second | 2.79 times |
The DSpark speed profile used a 16,384-token pool and retained 3.22 GB of GPU memory after initialization and CUDA graph capture.
Checkpoint
| Field | Value |
|---|---|
| Source revision | 404ea47aaa5d8a8b00049c9e9750089aca011ab2 |
| Format | ModelOpt NVFP4 safetensors |
| Export size | 20,593,147,464 bytes |
| Architecture | Qwen3_5ForConditionalGeneration |
| Native MTP tensors | 15 |
| Recommended runtime | SGLang Qwen3.8 image |
| Recommended hardware | NVIDIA RTX 5090 32 GB |
Run with the matched DSpark drafter
docker run --rm --gpus all --ipc=host -p 30000:30000 \
lmsysorg/sglang:qwen38-27b \
sglang serve \
--model-path fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 \
--quantization modelopt_fp4 \
--trust-remote-code \
--speculative-algorithm DSPARK \
--speculative-draft-model-path fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090 \
--speculative-draft-model-quantization unquant \
--speculative-dspark-block-size 7 \
--speculative-num-steps 1 \
--speculative-eagle-topk 1 \
--context-length 65536 \
--max-total-tokens 16384 \
--max-mamba-cache-size 1 \
--kv-cache-dtype fp8_e4m3 \
--attention-backend flashinfer \
--chunked-prefill-size 1024 \
--mamba-radix-cache-strategy extra_buffer_lazy \
--mamba-ssm-dtype bfloat16 \
--cuda-graph-max-bs-decode 1 \
--mem-fraction-static 0.90 \
--max-running-requests 1 \
--host 0.0.0.0 \
--port 30000
Provenance
- Base model: Qwen/Qwen3.8-27B
- BF16 source: orcarouter/Qwen3.8-27B-Uncensored
- Quantization: NVIDIA ModelOpt
0.47.0.dev70, commitc4129b6e - Calibration: 126 tokenized UltraChat conversations
- Validation: complete shard, tensor, dtype, MTP, load, API, and benchmark checks
Use
This checkpoint is intended for controlled local research, evaluation, and agent development. Use it responsibly and comply with the Apache 2.0 license and applicable law.