license: other
license_name: qwen-community-license-1.0
library_name: basalt
pipeline_tag: text-generation
tags:

• moe
• qwen
• basalt
• quantized
• uncensored
• speculative-decoding

Qwen3.8-FN-Uncensored-Q4_K_M — Basalt pack

A ready-to-run .basalt pack for the Basalt inference engine
(a fork of Strata, MIT). This is an uncensored Q4_K_M build of
Qwen3.8-Flash-Next (125B-class mixture-of-experts, qwen4exp architecture), converted from GGUF and
rebuilt with tuned speculative-decoding assets.

File

What is inside

• Experts: Q4_K_M, 48 layers, 512 routed experts per layer (24,576 expert slots in the routing profile).
• Output head: Q6_K (output.weight, 2,560 × 248,320).
• MTP / speculative decoding: mtp/dense marked tuning = self_distilled, mtp/draft_vocab = true,
  draft expert_format = q2_0. The draft head and draft vocabulary are the tuned ones, not the published defaults.
• Vision: vision/mmproj (907,543,296 bytes) plus vision/text_vocab, so image input works when the
  engine runs with --vision.
• Tokenizer: a full Qwen tokenizer directory and chat template are embedded in the pack; no external files are needed.
• Sections: tokenizer/*, dense, dense.index, experts, experts.layout, mtp/{dense,index,experts,draft_vocab},
  profile, vision/*, gguf, meta.

Provenance

Converted with Basalt's tools/convert/convert.py (git 2ef470c) from:

Each source's fingerprint (sha256 of the first and last 64 MiB plus the file size) is recorded verbatim in the
pack's meta section. The pack also carries the Qwen Community License 1.0 text and the model's own sampling
defaults: temperature 1.0, top_p 0.95, top_k 20.

Running it

Requirements: an NVIDIA Blackwell GPU (compute capability 12.0) and CUDA 13, plus enough system RAM for the
experts that do not fit in VRAM. Upstream targets Linux; a native Windows port exists in this fork's sibling
work. Use an engine build that has the Q4_K MMQ tiles (upstream 0e1f926 or later).

Flags this pack was measured with:

--max-context 524288 --kv int8 \
--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 262144 \
--spec 7 --spec-min-p 0.5 \
--expert-cache auto --prefill auto --resident-experts --vram-reserve-mib 700 \
--ple-io ram --kv-resident 32768 --vision dynamic

--mtp-max-t should stay unset: a nonzero value disables the suffix drafter.

Development A/B on one RTX 5090 (575 W power limit): ~220 tok/s decode at a 512k context window,
~5,900 tok/s prefill. Numbers are labelled with their power limit and are development measurements,
not headline figures.

License and attribution

The pack records the Qwen Community License 1.0 as its license; the license text is embedded in the
.basalt file itself. The engine is Basalt, a fork of
Strata (MIT). This repository is an independent conversion of
third-party uncensored weights — check the upstream model's terms before redistributing or deploying it
commercially.

This repository is currently private and under test.