license: apache-2.0
base_model: ukisai/Swift-Qwen3.8-Flash-Next
tags:
- gguf
- qwen
- qwen3.8
- qwen4exp
- swift
- mixture-of-experts
- quantized
- mixed-precision
- uncensored
- strata
- xolotl
- consumer-gpu
language: - en
- zh
- code
pipeline_tag: text-generation
Qwen3.8-Flash-Next · Swift 1.5 · XQ 45 GB, abliterated
A 125B-parameter MoE (6B active) on one gaming PC, RTX 4090 (24 GB) + 90 GB RAM + NVMe:
199 tok/s decode and 3,204 tok/s prefill at a 16K prompt, served at 256K context.
Runs on Xolotl, an engine built for this model and this class of
machine, on top of Strata.
Built from Swift 1.5, UkisAI's reasoning-efficient fine-tune of Qwen3.8-Flash-Next, with a refusal-direction
edit applied to the full-precision weights before quantisation.
Files
| file | what |
|---|---|
Qwen3.8-Flash-Next-Swift-1.5-XQ-45GB-Uncensored-00001-of-00002.gguf, -00002-of-00002.gguf |
the model, split for the Hub's 50 GB file limit (shard 1 holds the PLE table) |
pack-qfn45/ |
the Xolotl engine pack for these split files (rebuild it with tools/iq_pack.py if you merge the shards) |
strata-qfn45.json |
serve config (port 8099) |
GATES.md |
every measurement below, in full |
| file size | 73.7 GB (44.9 GB without the PLE table) |
| routed experts | 39.4 GB, 2.61 bits per weight on average |
| PLE n-gram table | 28.8 GB, read from disk with O_DIRECT |
| dense backbone | 5.5 GB, bit-identical to the Q8_0 parent |
| arch / tensors | qwen4exp / 1224 |
Expert formats: Q2_0 12.7 GB, IQ4_XS 10.7 GB, IQ1_M 9.9 GB, IQ4_NL 5.2 GB, Q8_0 0.9 GB.
Use Xolotl. Other runtimes have not been tested on this file.
How it was made
This is the first release of our XQ pipeline on this architecture, which is the release
V1 (below) promised:
- a teacher chain on the full-precision parent, so every layer is quantised against the inputs it really sees;
- per-expert, per-tier error screens on held-out routed tokens;
- a calibration-driven allocator that spends bits where the measured loss per byte is highest;
- GPTQ-family rounding with refinement on every routed expert. V1 used round-to-nearest.
The allocation method, screens and tooling are not published.
Quality
Full tables: GATES.md. V1 = our previous build (xol-qfn, 55.8 GB without PLE, plain
Qwen3.8-Flash-Next base).
Gauntlet v2: agentic coding, 7 tasks, 10 turns, tools in a sandbox, hidden tests graded after every turn,
thinking on.
| model | solved | SCORE | mean solve turn | wall time | decode during the run |
|---|---|---|---|---|---|
| this (45 GB) | 5/7 | 6.68 | 4.6 | 449 s | 164 tok/s |
| V1 xol-qfn (55.8 GB) | 6/7 | 6.50 | 5.8 | 856 s | 131 tok/s |
Qwen API qwen/qwen3.8-flash |
5/7 | 5.50 | 6.6 | 2,021 s | n/a |
This is the first local run to solve task 03 (the ray tracer's scene B). Every other run, the API included, stops
at 0.50 there. The two misses are narrow: task 01 reached 0.77 and task 06 reached 0.91. Each model ran once, so
5/7 against 6/7 is inside run-to-run variance.
Text, teacher-forced (5 segments × 999 positions, the same text and settings for both):
| perplexity V1 | perplexity 45 | next-token acc V1 | next-token acc 45 | |
|---|---|---|---|---|
| code | 2.072 | 2.044 | 83.4% | 84.4% |
| agent transcript | 4.630 | 4.457 | 72.5% | 74.1% |
| multilingual | 2.746 | 2.767 | 76.3% | 76.4% |
| docs | 10.585 | 11.026 | 48.5% | 47.9% |
| English prose | 4.437 | 5.070 | 58.9% | 56.7% |
| all | 4.154 | 4.264 (+2.6%) | 67.9% | 67.9% |
Code and agent text are slightly better than V1 and English prose is 14% worse in perplexity. The two builds
come from different fine-tunes, so part of that difference is Swift itself and not quantisation.
Against the full-precision parent (the abliterated Swift 1.5 Q8_0 this file was quantised from; the same 5 x 999
positions; top-1 agreement = same argmax as the Q8, KL over the Q8's top 64 tokens):
| file | size w/o PLE | top-1 agreement | KL (lower is better) | perplexity (Q8: 4.080) |
|---|---|---|---|---|
| this (XQ 45 GB) | 44.9 GB | 83.5% | 0.243 | 4.264 (+4.5%) |
| SC117 GSQ-RCO IQ3_XXS abliterated | 47.0 GB | 81.8% | 0.287 | 4.208 (+3.1%) |
| V1 xol-qfn (Qwen3.8-Flash-Next base, RTN) | 55.8 GB | 86.3% | 0.163 | 4.154 (+1.8%) |
| segment | agreement, this | SC117 | V1 |
|---|---|---|---|
| code | 92.0% | 90.3% | 93.2% |
| docs | 73.4% | 67.8% | 75.3% |
| English prose | 76.0% | 80.7% | 84.8% |
| multilingual | 92.0% | 87.4% | 92.2% |
| agent transcript | 84.2% | 83.0% | 86.0% |
At 2.1 GB smaller, this file tracks the parent more closely than the SC117 build of the same Swift 1.5 weights
(+1.7 points top-1, 15% lower KL); English prose is its weak segment. SC117's file is
SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF:
an abliterated build of UkisAI's GSQ-RCO quantisation (Swift-specific GSQ refinement on ISTA-DASLab's GSQ-RCO per-tensor
allocation profiles). V1 comes from a different base, so its column mixes base and quantisation differences.
Refusals. The abliteration survives quantisation: 0 of 100 held-out harmful prompts (JBB) refused, and
0 of 30 safe controls refused. The parent Q8 also scored 0/100.
Speed
Linux, RTX 4090 24 GB, Ryzen 7 5700X, 90 GB DDR4-3200, model on NVMe. Served config (strata-qfn45.json,
256K context, int8 KV, --spec 4 with the MTP drafter), fresh prompt, 1,024 generated tokens. Both files
were measured back to back with the same config:
| prompt | prefill (this) | decode (this) | prefill (V1) | decode (V1) |
|---|---|---|---|---|
| 16,000 | 3,204 tok/s | 199 tok/s | 3,231 tok/s | 124 tok/s |
| 8,000 | 2,819 tok/s | 160 tok/s | 2,740 tok/s | 128 tok/s |
Decode is bound by RAM bandwidth once the expert arena is warm, so it scales with memory speed more than with
the GPU.
Quick start
git clone https://github.com/Indras-Mirror/Xolotl.git xolotl && cd xolotl
bash tools/xolotl/fetch-llama.sh
cmake -S . -B build-xolotl -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DSTRATA_ENABLE_CUDA=ON -DSTRATA_GGML_CUDA=ON -DSTRATA_MMQ_KQUANTS=ON \
-DCMAKE_CUDA_ARCHITECTURES=89 -DSTRATA_GGML_DIR=$PWD/third_party/llama.cpp
ninja -C build-xolotl strata
hf download IndrasMirror/Qwen3.8-Flash-Next-Swift-1.5-XQ-Uncensored --local-dir qfn45
Then point strata-qfn45.json's paths at the download, and its --expert-profile and --mtp at your Xolotldata/ checkout. Serve it with python -m serve.server --engine strata --config strata-qfn45.json.
V1
xol-qfn (Qwen3.8-Flash-Next, RTN rounding, 84.6 GB) was measured but its weights were never uploaded. Its
numbers are kept above for comparison.
Credits
- Qwen for Qwen3.8-Flash-Next (Apache-2.0).
- UkisAI for Swift 1.5, the fine-tune this file is built from.
- mrfakename for the rank-2 refusal-direction edit style.
- Niko1221 for Strata, the tiered MoE engine this builds on.
- ggml / llama.cpp for the compute backend.
- Abliteration, quantisation, engine port and measurements: Indra's Mirror (Xolotl).
Support
If this has been useful and you want to help keep it going, you can
buy me a coffee.