base_model: ukisai/Swift-1.5-Qwen3.8-27b
base_model_relation: finetune
license: other
license_name: swift-open-license-1.0
license_link: LICENSE
quantized_by: sirfyyn
library_name: gguf
pipeline_tag: text-generation
tags: [gguf, llama.cpp, imatrix, qwen3_8, abliterated, uncensored, german, dspine, speculative-decoding]
Kate - Qwen 27B - GGUF - SpedUp - Abliterated
Run it with FLE Link: flelink run kate-27b-dspine installs the FLE engine, checks that Kate
fits your machine and downloads it. Kate's recipe uses FLE v1.3.0 (engine line dense); you need FLE Link 0.1.16 or newer. Measured it on your hardware? flelink bench --task pagoda --url http://127.0.0.1:8080 --share
opens a filled-in post for you to read and send.
Kate is Katy with the refusal direction removed from the weights.
Katy is my quantization of ukisai/Swift-1.5-Qwen3.8-27b that runs my
home agent around the clock on one RTX PRO 4500 Blackwell (32 GB). I removed the refusal direction because an agent that
manages its own machine kept refusing ordinary system work. It is a private project; I publish it as it is.
The quant runs with llama.cpp. I measure and serve it with my fork FLE, the engine behind
FLE Link, together with the DSpine draft head included here.
Warning. Kate no longer refuses requests that the original model refused for safety reasons. You are responsible
for what you do with it. Do not put it in front of people who expect a safety-tuned assistant.
Files
| File | Size | What it is |
|---|---|---|
kate-27b-IQ4_XS-imat-R1-A.gguf |
15.48 GB (15,475,951,648 B) | Kate, IQ4_XS in bartowski's tensor layout with my importance matrix A (recipe R1-A, below). The file I run. |
Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf |
1.17 GiB (1,257,892,896 B) | DSpine block-16 draft head. Not my work: unchanged weights of tencent/Qwen3.8-27B-DSpine-b16, Apache-2.0, as GGUF Q4_K_M by aj9o9, redistributed unchanged. |
kate-imatrix-A.gguf |
27.3 MB (27,300,192 B) | Importance matrix A, computed on Kate's BF16 from public calibration text only. |
SHA-256 per file: see SHA256SUMS. Kate: 4bd778cf6912cc1f85fa1aea66503a2d2b3dd7e9d2e58e48255038dbdcadae0b,
imatrix A: ab9bb9b29a02ee97f00061ed7c3a78757271411158c6b3a0ad331af23d95f9d5.
How Kate was made
- Activations of the residual stream after every layer, at the last token of the chat template, for 178 harmful and
178 harmless prompts (English and German). - Refusal direction per layer = mean(harmful) − mean(harmless). The layer with the best separation and a direction that
agrees between English and German was chosen: layer 45; it was removed from block 30 onward. - That one direction removed from every matrix that writes into the residual stream (attention output, DeltaNet output,
FFN down projection, token embedding), in BF16: W′ = W − d(dᵀW). - Importance matrix A from public calibration text (no private data, 254,805 tokens), computed on Kate's BF16, then
IQ4_XS in bartowski's tensor layout (8-bit attention K/V, 6-bit attention output, more bits for the DeltaNet
output, F32 DeltaNet alpha/beta; paid for by 4-bit DeltaNet qkv). I chose it on a new measurement basis: eight kinds
of German and English text (prose, technical docs, dialogue, code, math, agent traces) in three disjoint, hashed sets
for calibration, selection and one final test. Nine candidates (my plain IQ4_XS as in Katy, bartowski's layout, and
that layout with 6-bit ssm_out, each with three calibration sets) were compared on the selection set under a rule
written down beforehand; the winner was then tested once. A better tensor layout, not calibration data, closed the
gap to the best public quant: the layout alone lowered divergence by 3.4 %, the three calibration sets differ by 0.5 %.
Not done: healing the quant. I tried twice to heal the quantization by training only the fp16 super-block scales
against Kate's own 8-bit teacher, codes unchanged (same file size and speed). Both runs got better on text like their
training corpus and worse on foreign text:
| validation (own corpus) | Python reference | German Wikipedia | German Kubernetes docs | |
|---|---|---|---|---|
| run 1, public corpus | −32 % | −4 % | +4 % | +7 % |
| run 2, mixed corpus, rule fixed in advance | −25 % | −2 % (within error) | +11 % | +13 % |
(KL divergence against Kate's BF16; run 2 paired over sections, W and K worse by 5–7 times their error.) By the rule
written down before run 2, every test text had to improve. It did not, so Kate ships unhealed. Details and what I
would change are in the report.
Method after Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (arXiv:2406.11717).
Refusals
Held-out test prompts, measured on this file, refusal counted by phrase list in the first 160 characters (coarse; raw
answers kept), Wilson 95 %:
| Set | Katy | Kate (this file) |
|---|---|---|
| harmful, English (100) | 98 / 100 (93.0–99.4 %) | 70 / 100 (60.4–78.1 %) |
| harmful, German (34) | 22 / 34 (47.9–78.5 %) | 8 / 34 (12.4–40.0 %) |
| harmless, English (100) / German (34) | 3 / 100 (1.0–8.5 %) / 0 / 34 (0–10.2 %) | 2 / 100 (0.6–7.0 %) / 0 / 34 (0–10.2 %) |
| XSTest, safe-but-sounds-unsafe (250) | 15 / 250 (3.7–9.7 %) | 4 / 250 (0.6–4.0 %) |
| OR-Bench hard (100) | 88 / 100 (80.2–93.0 %) | 1 / 100 (0.2–5.4 %) |
| borderline, German (40) | 3 / 40 (2.6–19.9 %) | 0 / 40 (0–8.8 %) |
So Kate still refuses seven out of ten clearly harmful English prompts; what went away is the over-refusal. Every
Kate rate is inside the interval of the earlier Kate file (same weights, plain IQ4_XS), so the new quant recipe did not
move refusals.
Quality
KL divergence against BF16 per kind of text (final test set of the measurement basis, 27–29 sections of 2,048 tokens
per kind; Kate against Kate's BF16, the public quants against Katy's BF16; mean ± standard error):
| Kind | Kate, this file (15.48 GB) | Kate, plain IQ4_XS (15.31 GB) | bartowski IQ4_XS (15.48 GB) | UkisAI IQ4_XS (15.48 GB) | UkisAI Q4_K_M (17.44 GB) |
|---|---|---|---|---|---|
| German prose | 0.01557 ± 0.00034 | 0.01714 | 0.01567 | 0.01556 | 0.01262 |
| English prose | 0.01502 ± 0.00041 | 0.01665 | 0.01437 | 0.01448 | 0.01320 |
| German technical docs | 0.01894 ± 0.00033 | 0.02110 | 0.02001 | 0.01995 | 0.01557 |
| English technical docs | 0.02082 ± 0.00068 | 0.02100 | 0.02056 | 0.02062 | 0.01634 |
| code | 0.01114 ± 0.00023 | 0.01253 | 0.01114 | 0.01125 | 0.00890 |
| German dialogue | 0.25136 ± 0.00795 | 0.26775 | 0.25494 | 0.25519 | 0.21366 |
| math | 0.04268 ± 0.00275 | 0.04729 | 0.04195 | 0.04038 | 0.03674 |
| agent traces | 1.42618 ± 0.02400 | 1.49850 | 1.44692 | 1.45086 | 1.39745 |
| mean KLD | 0.22521 | 0.23775 | 0.22820 | 0.22854 | 0.21431 |
| mean top-1 agreement | 92.70 % | 92.34 % | 92.66 % | 92.66 % | 93.12 % |
Paired over sections against the plain IQ4_XS Kate file: better on 6 of 8 kinds, equal within error on English
technical docs and math, worse on none; top-1 agreement up on every kind; mean KLD −5.3 %. Against their own BF16,
Kate is level with bartowski's and UkisAI's IQ4_XS of the parent model (different references, so read it as parity).
Caveat: llama-perplexity in my fork does not parse special tokens, so the chat kinds (dialogue, math, agent) are
tokenized literally. That is the same for every file in the table, but not how the model sees them in use; it is also
why their absolute numbers are high.
Katy and Kate on the same tasks, same build:
| Katy | Kate | |
|---|---|---|
| KLD E / W / K (Python reference, German Wikipedia, German Kubernetes docs; against own BF16) | 0.0647 / 0.0128 / 0.0160 | 0.0612 / 0.0112 / 0.0141 |
| KLD E / W / K against Katy's BF16 (the price of abliteration plus quant) | 0.0647 / 0.0128 / 0.0160 | 0.0661 / 0.0116 / 0.0154 |
| ARC-Challenge (1172) | 94.3 % (92.8–95.5) | 94.2 % (92.7–95.4) |
| GSM8K (250) | 97.2 % (94.3–98.6) | 97.6 % (94.9–98.9) |
| LiveCodeBench (100), no thinking / thinking budget 8000 | 42 / 67 | 44 / 65 |
| Agent tasks (24, 8k–32k prompts): tool use / text | 0.393 / 0.399 | 0.446 / 0.429 |
| Structured output: JSON (40), long output (9), tool calls (40), agent loops (10) | 40 / 5 / 38 / 9 | 39 / 6 / 38 / 9 |
| Loops (responses stuck in repetition, all outputs) | 7 | 6 |
Speed
One RTX PRO 4500 Blackwell (32 GB), FLE, DSpine head with n-max 15 plus n-gram drafting, f16 KV cache, paired A-B-B-A.
Kate against Katy (Katy with its own trained head), paired: agent tasks 152.6 against 161.5 tok/s, −5.5 % (95 %: −18.2 to
+5.5, unclear); code (LiveCodeBench) 153.6 against 152.3 tok/s, +0.9 % (−4.0 to +5.9, within noise). For reference, Katy in real
agent use: median 154.6 tok/s over 1,729 responses.
Smallest card (memory limit simulated with -fitt, same build): 20 GB keeps all 66 layers on the GPU at full speed
(142.7 tok/s); 18 GB 111.9; 16 GB 37.3; 14 GB 22.8; 12 GB 14.1 tok/s.
Start command I use (FLE; upstream llama.cpp needs a DSpine-capable build):
llama-server -m kate-27b-IQ4_XS-imat-R1-A.gguf -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf --spec-type draft-dflash,ngram-mod \
--spec-draft-n-max 15 --spec-draft-p-min 0 --spec-draft-ngl all -ngl 99 -c 163840 -fa on -ctk f16 -ctv f16 \
--spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 32 --spec-ngram-mod-n-match 16 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0
Full report
Method, every measurement with intervals, negative results (including a benchmark bug in my own harness that had
inflated paired runs by up to 68 %): the report as a video: https://youtu.be/Bye6WPpkxA4 (the written version will be linked here).
Licence
Swift Open License v1.0 for the Swift contribution (see LICENSE, including the commercial-use threshold in §5),
Apache-2.0 for the Qwen3.8-27B base and the DSpine head (see LICENSE-APACHE-2.0), details in NOTICE.
