license: apache-2.0
library_name: ninfer
pipeline_tag: image-text-to-text
base_model: orcarouter/Qwen3.8-27B-Uncensored-NVFP4
tags:

• ninfer
• qwen3.8
• multimodal
• conversational
• uncensored
• abliterated
• nvfp4
• dflash2
• speculative-decoding
• mtp
• cuda
• rtx-5090
• image-text-to-text
  inference: false

Qwen3.8-27B Uncensored, NInfer artifact with DFlash2

This repository contains an NInfer v3 (.ninfer) artifact for
ninfer, an OpenAI-compatible inference
server that runs a Qwen3.8-27B vision-language model on a single
RTX 5090 (32 GB).

This artifact was converted from
orcarouter/Qwen3.8-27B-Uncensored-NVFP4
(the NVFP4 quantization of orcarouter's abliterated/uncensored Qwen3.8-27B),
and DFlash2 speculative-decoding support was added on top, with the draft
model from
z-lab/Qwen3.8-27B-DFlash2.

• Base checkpoint: orcarouter/Qwen3.8-27B-Uncensored-NVFP4 (revision f8489198)
• Draft model: z-lab/Qwen3.8-27B-DFlash2 (revision 50307d4c, 5-layer sliding-attention draft)
• Conversion recipe: qwen3_8_27b_nvfp4 via the ninfer-v3 converter
• License: Apache-2.0 (inherited from the source checkpoints)

Files

Requirements

• RTX 5090 32 GB (CUDA architecture sm_120a); any sm_120a GPU with 32 GB
  should work
• ninfer at or after revision
  9e163eee (the artifact was converted and is served by that revision,
  2026-09-18, which includes the custom Jinja chat-template commit 98dada0e)
• CUDA 13.0 toolchain on the conversion side (torch 2.14.1+cu130)

Serving

./build/apps/ninfer-serve qwen3_8_27b_uncensored_dflash2.ninfer \
  --max-context 200000 --kv-capacity 200000 --max-concurrency 1 \
  --kv-dtype k8v4 --device-state-slots 2 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft \
  --vision --preserve-thinking --port 8085

The server then exposes OpenAI-compatible endpoints under
http://127.0.0.1:8085/v1 with model id qwen3.8-27b.
The GPU uses roughly 31.6 GB of 32 GB at startup with these settings.

Measured behavior

On the serving build (600 completion tokens, reasoning_effort: none):
about 164 tokens/s, with DFlash2 draft acceptance of 387/1475
(roughly 26%). Abliteration behavior is preserved: the model answers
prompts the official censored checkpoint refuses.

Tokenizer notes

The tokenizer files in the orcarouter checkpoint required adjustments to pass
ninfer's frontend gates. The fixed files are included under reference/ and
are what was baked into this artifact:

1. tokenizer.json - the Split pattern was replaced with ninfer's built-in
   Qwen split pattern (including \p{M}).
2. tokenizer_config.json - a 33-entry added_tokens_decoder
   (ids 248044-248076) was added, and add_bos_token: false /
   add_prefix_space: false were made explicit.

The chat template is ninfer's built-in Qwen3.8 template
(reference/chat_template.jinja). It is byte-identical to the template
updated in neroued's 1cbd84e ("update the built in qwen chat template") and
is a strict superset of orcarouter's plain-Qwen template: it additionally
handles the developer role, final-assistant message continuation (used by
the DFlash2/MTP draft path), tool-call histories whose user turn was removed,
and non-leading system messages.

Verification

Check the artifact checksum:

sha256sum qwen3_8_27b_uncensored_dflash2.ninfer
# 4518e669f7e3c224c10c21c78abaa28ad5304cc22fed4777a73bba4f841416cf