license: other
license_name: swift-open-license-1.0
base_model: ukisai/Swift-1.5-Qwen3.8-27b
base_model_relation: quantized
tags:
- gguf
- qwen
- qwen3.8
- qwen3.5
- swift
- dense
- quantized
- mixed-precision
- uncensored
- abliterated
- vision
- xolotl
- consumer-gpu
language: - en
- zh
- code
pipeline_tag: image-text-to-text
Qwen3.8-27B · Swift 1.5 · XQ 14 GB, abliterated
A 27B dense reasoning model with vision on one 24 GB card: 200K context, MTP speculative decoding and the vision
encoder all resident on an RTX 4090, peak 23.7 GB.
Built from Swift 1.5, UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B, with a refusal-direction edit applied
to the full-precision weights before quantisation, then quantised with Indra's Mirror's mixed-precision XQ.
Gauntlet v3-small: 6/6 tasks solved, score 6.00, the best 27B result on our machine so far (d0xin's IQ3_S build:
5/6, 5.50), at ~120 tok/s decode during the tasks.
Files
| file | what |
|---|---|
Qwen3.8-27B-Swift-1.5-XQ-14GB-Uncensored.gguf |
the model (14.18 GB), arch qwen35, 866 tensors, MTP block included |
mmproj-Qwen3.8-27B-BF16.gguf |
vision projector (0.93 GB), from Qwen3.8-27B, unchanged |
xolotl-27b-swift-xq.json |
Xolotl serve config: 200K, k8v4 KV, MTP 3 + n-gram, vision |
GATES.md |
every measurement below, in full |
| layer matrices | 11.63 GB, 3.82 bits per weight on average |
| formats | IQ4_XS 5.79 GB, IQ3_XXS 1.98, IQ3_S 1.73, Q3_K 1.11, Q4_K 0.58, IQ4_NL 0.29, Q5_K 0.07 |
| token embedding / output head | Q6_K / Q6_K (1.04 GB each) |
| MTP block | Q8_0 (0.45 GB) |
Every tensor is a standard ggml type, so the file runs in llama.cpp as well as Xolotl. The quality numbers
below were measured with llama.cpp, the agent and serving numbers with Xolotl.
How it was made
- Abliteration of Swift 1.5 in full precision: an ARA search (per-module LoRA, layers 15-58) with
Heretic, merged into the BF16 weights. - XQ, Indra's Mirror's own quantisation pipeline (private): every layer matrix gets its own format and
bit-width from a measured budget, fitted against the abliterated model.
The abliteration goes in before quantisation, so the quant is fitted to the model you actually run.
Quality
Measured against the abliterated Swift 1.5 Q8_0 this file was quantised from, with llama-perplexity --kl-divergence, on two sets:
- short: 8 x 4,096 tokens of held-out text
- 32K agent: 3 x 32,768 tokens of real agent transcripts (tool calls, file reads, code) in the model's own chat
template, the kind of context this model is served for
Lower KL = closer to the Q8_0. "Same top" = how often the quant's most likely next token is the Q8_0's.
| file | size | short: KL / same top / ppl ratio | 32K agent: KL / same top / ppl ratio |
|---|---|---|---|
| Q4_K_M (llama.cpp, imatrix) | 16.81 GB | 0.805 / 84.7% / x1.44 | 2.60 / 73.2% / x3.10 |
| this (XQ 3.8) | 14.47 GB | 0.881 / 83.8% / x1.47 | 3.02 / 70.1% / x2.73 |
| XQ 3.53 (not uploaded) | 13.65 GB | 1.008 / 81.9% / x1.42 | 3.30 / 67.8% / x2.96 |
| d0xin IQ3_S (Swift 1.5 uncensored) | 13.20 GB | 1.517 / 77.4% / x1.63 | 4.26 / 63.2% / x2.75 |
(Sizes in this table are the evaluated files: the uploaded file is the same model with its token embedding at Q6_K
instead of Q8_0, 0.3 GB smaller.)
- 2.3 GB smaller than Q4_K_M, close behind it on the short set, and the best of the four on the 32K agent ratio.
- Against d0xin's IQ3_S, the size class most people run on 24 GB: 42% lower KL on the short set and 29% lower
on the 32K set. d0xin's file comes from its own uncensored Swift 1.5, so part of its gap is a different parent. - Absolute KL is high for every file here, and the 32K set is hard for all of them; only the comparison within
one table means anything.
Behaviour gates (abliterated Q6_K before quantisation -> this file):
| Q6_K parent | this | |
|---|---|---|
| harmful prompts refused (of 100) | 0 | 0 (one disclaimer flagged by the classifier, answered in full) |
| harmless prompts refused (of 100) | 0 | 0 |
| says "I don't know" on unanswerable questions (of 30) | 21 | 19 |
| factual questions correct (of 30) | 28 | 29 |
| vulnerable code caught (of 20) / clean code flagged (of 20) | 20 / 2 | 20 / 2 |
| short reasoning problems correct (of 48) | 48 | 48 |
| thinking tokens, mean / median | 375 / 154 | 455 / 192 |
Gauntlet v3-small: 6 hard agentic coding tasks, tools in a sandbox, hidden tests graded after every turn, 10 turns,
thinking on (32K budget), each model as served on Xolotl (200K context, MTP 3 + n-gram):
| model | size | solved | SCORE | tokens | wall time | decode during the run |
|---|---|---|---|---|---|---|
| this (XQ 3.8) | 14.2 GB | 6/6 | 6.00 | 73,190 | 10.4 min | ~120 tok/s |
| XQ 3.53 (same pipeline, not uploaded) | 13.4 GB | 3/6 | 4.44 | 127,170 | 19.6 min | 111.9 tok/s |
| d0xin IQ3_S (Swift 1.5 uncensored) | 13.2 GB | 5/6 | 5.50 | 116K | 104.6 tok/s | |
| our earlier Qwen3.8-27B XQ (plain base, 4.5 bpw) | 16.2 GB | 4/6 | 4.97 | 153K | 26 min | 99.6 tok/s |
Solved at turn 5 / 3 / 5 / 4 / 7 / 3 (tasks 01 / 03 / 04 / 06 / 09 / 10). Task 04 had not been solved by any earlier
27B run. The engine's sampler is seeded, so a repeated run gives the same tokens (this one was run three times:
73,190 tokens each, 118-123 tok/s): every row is one sample per model, not an average over random runs.
Speed and fit
Linux, RTX 4090 24 GB, Ryzen 7 5700X, 110 GB DDR4, model on NVMe.
Xolotl, the served config (xolotl-27b-swift-xq.json: 200K context, k8v4 KV, MTP 3 + n-gram, vision on),
fresh prompt, 256 generated tokens:
| prompt | prefill | decode |
|---|---|---|
| 6,000 | 2,091-2,121 tok/s | 101-115 tok/s |
| 16,000 | 1,837-1,864 tok/s | 114-126 tok/s |
The 200K KV cache, the MTP head and the vision encoder all stay in VRAM: peak 23.7 GB of 24.6 after three image
requests (up to 6,144 image tokens) and a 16K-token prompt, including ~0.7 GB the desktop uses. With anything else on
the card, add --kv-resident auto --vram-reserve-mib 2400 and the KV past ~100K moves to pinned RAM.
llama.cpp (llama-bench, plain decoding, no speculative decoding): prefill 3,679 tok/s at 2,048 tokens,
decode 59.0 tok/s.
Quick start
llama.cpp
hf download IndrasMirror/Qwen3.8-27B-Swift-1.5-XQ-Uncensored --local-dir swift-xq
llama-server -m swift-xq/Qwen3.8-27B-Swift-1.5-XQ-14GB-Uncensored.gguf \
--mmproj swift-xq/mmproj-Qwen3.8-27B-BF16.gguf -ngl 99 -c 131072 -fa on -ctk q8_0 -ctv q8_0 --jinja \
--temp 1.0 --top-p 0.95 --top-k 20
Xolotl (MTP speculative decoding, 200K, vision)
git clone https://github.com/Indras-Mirror/Xolotl.git xolotl && cd xolotl
bash tools/xolotl/fetch-llama.sh
cmake -S . -B build-xolotl -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DSTRATA_ENABLE_CUDA=ON -DSTRATA_GGML_CUDA=ON -DSTRATA_MMQ_KQUANTS=ON \
-DCMAKE_CUDA_ARCHITECTURES=89 -DSTRATA_GGML_DIR=$PWD/third_party/llama.cpp
ninja -C build-xolotl xolotl-dense
hf download IndrasMirror/Qwen3.8-27B-Swift-1.5-XQ-Uncensored --local-dir swift-xq
Point the paths in xolotl-27b-swift-xq.json at your checkout and the download, thenpython -m serve.server --engine strata --config xolotl-27b-swift-xq.json --port 8090, and use any OpenAI- or
Anthropic-compatible client on http://127.0.0.1:8090/v1.
Recommended sampling (Swift 1.5 / Qwen3.8): temperature 1.0, top_p 0.95, top_k 20, thinking on.
Credits
- Qwen for Qwen3.8-27B and its vision projector (Apache-2.0).
- UkisAI for Swift 1.5, the fine-tune this file is built
from; its licence (Swift Open License 1.0) applies. - Philipp Emanuel Weidmann for Heretic, whose ARA search found the refusal edit.
- d0xin for the Swift 1.5 IQ3_S build used as the comparison point.
- perronemirko for the Qwen3.8-27B dense port in Xolotl, Niko1221 for Strata,
ggml / llama.cpp for the compute backend and the evaluation tools. - Abliteration, quantisation, engine and measurements: Indra's Mirror (Xolotl).
Support
If this has been useful and you want to help keep it going, you can
buy me a coffee.