license: apache-2.0
base_model:
- just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq
- orcarouter/Qwen3.8-27B-Uncensored
- z-lab/Qwen3.8-27B-DFlash2
pipeline_tag: image-text-to-text
tags: - radiance
- rocm
- amd
- rdna4
- mxfp4
- speculative-decoding
- dflash2
- vision
- abliterated
- uncensored
Qwen3.8-27B Uncensored MXFP4 + DFlash2 — radiance container
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq
— orcarouter/Qwen3.8-27B-Uncensored, an
abliterated Qwen3.8-27B, quantised to AWQ-calibrated OCP-MXFP4 with AMD Quark — as a single .rad
container for the radiance inference engine (AMD RDNA4, ROCm), with its vision tower and the
z-lab/Qwen3.8-27B-DFlash2 block-diffusion
drafter merged in for speculative decoding.
This model's safety alignment has been removed: it answers requests the original model refuses.
The source cards describe how it was made, how it was evaluated and what it is for.
This is the uncensored counterpart of
StillDeadcode/qwen3.8-27b-mxfp4 (same
quantisation and recipe, the censored AMD trunk) and of
StillDeadcode/qwen3.8-27b-uncensored-fp8
(same uncensored weights at FP8, 28.67 GiB). 18 GiB total means it serves one 32 GB card.
Needs radiance 1.5.0 or newer
| File | qwen3.8-27b-uncensored-mxfp4-df2.rad — 18.05 GiB, RAD2, 1,101 tensors |
| Weights | the release's MXFP4 trunk exactly as it ships — e2m1 codes and an E8M0 exponent per 32, byte for byte, nothing re-quantised — served at 4-bit weights against E4M3 activations; the lm_head made block FP8 by the recipe |
| Speculator | DFlash2 (5 blocks, drafting over target layers 5/19/33/47/61, block size 8), its linears OCP-MXFP4 and its vocabulary head as 2-bit codes, whose top 32 candidates the engine rescores against the model's own FP8 head. Drafts are verified, so they change the speed and never the output |
| Vision | the 27-block vision tower, bf16, as it ships: images and video in chat requests (text generation is what was exercised on this file) |
| Context | 262,144 tokens trained |
Speed
Radeon AI PRO R9700 at 250 watts (gfx1201), one card (--tp 1), radiance 1.5.0,--max-num-seqs 8 --max-model-len 260000 --kv-cache-dtype fp8; BetterBench v1.0 corpus, greedy,
5 passes per category, cold prefix cache:
| combined weighted decode (code 0.3 / reasoning 0.2 / prose 0.15 / json 0.15 / file_edit 0.1 / summarization 0.1) | 132.1 tok/s (median), TTFT p50 126 ms |
| prose / code (tok/s) | 88.8 / 81.5 |
| json / math / file_edit (tok/s) | 193.6 / 195.3 / 195.9 |
| prefill, 2k / 8k / 64k unique prompt (tok/s, TTFT p50) | 2,837 (0.53 s) / 2,895 (2.03 s) / 2,450 (19.2 s) |
| draft acceptance, depth 7 | 53.8 % (the engine's own /metrics counters, live traffic) |
Concurrency sweep, 48 mixed requests, cold prefix — 48/48 succeeded at every level:
| N | aggregate tok/s | TTFT p50 (ms) |
|---|---|---|
| 1 | 135.4 | 59.0 |
| 2 | 235.8 | 91.5 |
| 4 | 388.5 | 109.0 |
| 8 | 492.1 | 151.7 |
The drafter is doing its work here: the same drafter merged onto a different 27B MXFP4 merge accepted
~10 % of drafts, and 53.8 % at depth 7 is what turns ~35 tok/s of raw bandwidth-bound decode into the
figures above.
14.84 GiB of static weights on a 31.86 GiB card leaves ~15.7 GiB of KV: 262,144 token max single request (model limitation), 433,000 max tokens parallel requests.
Serve
docker compose -f ~/radiance-compose/q38-unc-mxfp4-df2.yaml up -d
radiance --model qwen3.8-27b-uncensored-mxfp4-df2.rad --tp 1 --max-model-len 260000 \
--kv-cache-dtype fp8 --max-num-seqs 8 --host 0.0.0.0 --port 8000 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repetition-penalty 1.0
--tp 2 on two cards is fine and faster; --vision-tower-placement ram keeps the vision tower out
of the way of the KV pool on a single card.
The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls and
structured output, and image_url / video_url parts in chat messages. The drafter's depth is
chosen automatically (this drafter's operating point is 7; --num-speculative-tokens N states one,0 turns speculation off).
Requests that send no sampling parameters get the defaults above; they are what this model was
checked with.
How this file was made
rad-convert q38-unc-mxfp4-nomtp --draft-model Qwen3.8-27B-DFlash2-FP8 \
--recipe q38-27b-quark-mxfp4-df2.recipe -o qwen3.8-27b-uncensored-mxfp4-df2.rad
q38-unc-mxfp4-nomtp is the source checkpoint with one thing edited: a copy of config.json withquantization_config.layer_quant_config dropped, the weights themselves symlinked in. Quark wrote
that checkpoint's eight mtp.* tensors as fp8-e4m3 per-channel, and the MXFP4 importer demands OCP
4-bit throughout, so a descriptor for tensors nothing reads was vetoing the whole checkpoint — the
DFlash2 drafter displaces the shipped MTP head. Consequence: this container has no native MTP
head, so a drafter A/B against it is not available; DFlash2 is the only speculator.
Built and tested on 1× Radeon AI PRO R9700 (gfx1201), radiance 1.5.0, ROCm.
Limitations
- Uncensored/abliterated: it will produce output the base model refuses. You are responsible for how you use it.
- MXFP4 is aggressive; the trunk's quality is the source release's, and the lm_head was re-expressed as block FP8 by the recipe.
- Not tested for production or safety-critical use.