← back to catalog · registered 2026-11-16 20:03

sirfyyn/Kate-Qwen-27B-GGUF-SpedUp-Abliterated

sirfyyn Qwen 27B GGUF
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/sirfyyn%2FKate-Qwen-27B-GGUF-SpedUp-Abliterated"
Response includes
  • classification m8
  • files 9
  • author_summary 1 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
329
Likes
0
Model age
5d ago
created 2026-11-16
Downloads over time
Now 329 → from 0 ↑ 0%
0121241362 0 on Nov 16 329 on Nov 22 Nov
Nov 16 → Nov 22 · 7 snapshots · spans 6 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
other
Quantizations
Q4_K
Tags
gguf llama.cpp imatrix qwen3_8 abliterated uncensored german dspine speculative-decoding text-generation arxiv:2406.11717 base_model:ukisai/Swift-1.5-Qwen3.8-27b

Related

Total size
15.6 GB
Files
9
Quantizations
2
Registered
2026-11-16 20:03
Last updated on HF
2026-11-16 19:24

Files by quantization

Drafts and companions 2 files 1.20 GB
Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf 1.17 GB 0c6c984a download
kate-imatrix-A.gguf 26.0 MB ab9bb9b2 download
Auxiliary files 7 files 14.4 GB
kate-27b-IQ4_XS-imat-R1-A.gguf 14.4 GB 4bd778cf download
LICENSE 13.0 KB 209a5720 download
LICENSE-APACHE-2.0 11.3 KB f938136e download
README.md 10.2 KB 22f6744f download
NOTICE 2.05 KB 301ca846 download
.gitattributes 1.67 KB 5be0a8bc download
SHA256SUMS 516 B ec369009 download

README current version from Hugging Face


base_model: ukisai/Swift-1.5-Qwen3.8-27b
base_model_relation: finetune
license: other
license_name: swift-open-license-1.0
license_link: LICENSE
quantized_by: sirfyyn
library_name: gguf
pipeline_tag: text-generation
tags: [gguf, llama.cpp, imatrix, qwen3_8, abliterated, uncensored, german, dspine, speculative-decoding]

Kate - Qwen 27B - GGUF - SpedUp - Abliterated

FLE Link: run Kate in one command

Run it with FLE Link: flelink run kate-27b-dspine installs the FLE engine, checks that Kate
fits your machine and downloads it. Kate's recipe uses FLE v1.3.0 (engine line dense); you need FLE Link 0.1.16 or newer. Measured it on your hardware? flelink bench --task pagoda --url http://127.0.0.1:8080 --share
opens a filled-in post for you to read and send.

Kate is Katy with the refusal direction removed from the weights.
Katy is my quantization of ukisai/Swift-1.5-Qwen3.8-27b that runs my
home agent around the clock on one RTX PRO 4500 Blackwell (32 GB). I removed the refusal direction because an agent that
manages its own machine kept refusing ordinary system work. It is a private project; I publish it as it is.

The quant runs with llama.cpp. I measure and serve it with my fork FLE, the engine behind
FLE Link, together with the DSpine draft head included here.

Warning. Kate no longer refuses requests that the original model refused for safety reasons. You are responsible
for what you do with it. Do not put it in front of people who expect a safety-tuned assistant.

Files

File Size What it is
kate-27b-IQ4_XS-imat-R1-A.gguf 15.48 GB (15,475,951,648 B) Kate, IQ4_XS in bartowski's tensor layout with my importance matrix A (recipe R1-A, below). The file I run.
Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf 1.17 GiB (1,257,892,896 B) DSpine block-16 draft head. Not my work: unchanged weights of tencent/Qwen3.8-27B-DSpine-b16, Apache-2.0, as GGUF Q4_K_M by aj9o9, redistributed unchanged.
kate-imatrix-A.gguf 27.3 MB (27,300,192 B) Importance matrix A, computed on Kate's BF16 from public calibration text only.

SHA-256 per file: see SHA256SUMS. Kate: 4bd778cf6912cc1f85fa1aea66503a2d2b3dd7e9d2e58e48255038dbdcadae0b,
imatrix A: ab9bb9b29a02ee97f00061ed7c3a78757271411158c6b3a0ad331af23d95f9d5.

How Kate was made

  1. Activations of the residual stream after every layer, at the last token of the chat template, for 178 harmful and
    178 harmless prompts (English and German).
  2. Refusal direction per layer = mean(harmful) − mean(harmless). The layer with the best separation and a direction that
    agrees between English and German was chosen: layer 45; it was removed from block 30 onward.
  3. That one direction removed from every matrix that writes into the residual stream (attention output, DeltaNet output,
    FFN down projection, token embedding), in BF16: W′ = W − d(dᵀW).
  4. Importance matrix A from public calibration text (no private data, 254,805 tokens), computed on Kate's BF16, then
    IQ4_XS in bartowski's tensor layout (8-bit attention K/V, 6-bit attention output, more bits for the DeltaNet
    output, F32 DeltaNet alpha/beta; paid for by 4-bit DeltaNet qkv). I chose it on a new measurement basis: eight kinds
    of German and English text (prose, technical docs, dialogue, code, math, agent traces) in three disjoint, hashed sets
    for calibration, selection and one final test. Nine candidates (my plain IQ4_XS as in Katy, bartowski's layout, and
    that layout with 6-bit ssm_out, each with three calibration sets) were compared on the selection set under a rule
    written down beforehand; the winner was then tested once. A better tensor layout, not calibration data, closed the
    gap to the best public quant:
    the layout alone lowered divergence by 3.4 %, the three calibration sets differ by 0.5 %.

Not done: healing the quant. I tried twice to heal the quantization by training only the fp16 super-block scales
against Kate's own 8-bit teacher, codes unchanged (same file size and speed). Both runs got better on text like their
training corpus and worse on foreign text:

validation (own corpus) Python reference German Wikipedia German Kubernetes docs
run 1, public corpus −32 % −4 % +4 % +7 %
run 2, mixed corpus, rule fixed in advance −25 % −2 % (within error) +11 % +13 %

(KL divergence against Kate's BF16; run 2 paired over sections, W and K worse by 5–7 times their error.) By the rule
written down before run 2, every test text had to improve. It did not, so Kate ships unhealed. Details and what I
would change are in the report.

Method after Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (arXiv:2406.11717).

Refusals

Held-out test prompts, measured on this file, refusal counted by phrase list in the first 160 characters (coarse; raw
answers kept), Wilson 95 %:

Set Katy Kate (this file)
harmful, English (100) 98 / 100 (93.0–99.4 %) 70 / 100 (60.4–78.1 %)
harmful, German (34) 22 / 34 (47.9–78.5 %) 8 / 34 (12.4–40.0 %)
harmless, English (100) / German (34) 3 / 100 (1.0–8.5 %) / 0 / 34 (0–10.2 %) 2 / 100 (0.6–7.0 %) / 0 / 34 (0–10.2 %)
XSTest, safe-but-sounds-unsafe (250) 15 / 250 (3.7–9.7 %) 4 / 250 (0.6–4.0 %)
OR-Bench hard (100) 88 / 100 (80.2–93.0 %) 1 / 100 (0.2–5.4 %)
borderline, German (40) 3 / 40 (2.6–19.9 %) 0 / 40 (0–8.8 %)

So Kate still refuses seven out of ten clearly harmful English prompts; what went away is the over-refusal. Every
Kate rate is inside the interval of the earlier Kate file (same weights, plain IQ4_XS), so the new quant recipe did not
move refusals.

Quality

KL divergence against BF16 per kind of text (final test set of the measurement basis, 27–29 sections of 2,048 tokens
per kind; Kate against Kate's BF16, the public quants against Katy's BF16; mean ± standard error):

Kind Kate, this file (15.48 GB) Kate, plain IQ4_XS (15.31 GB) bartowski IQ4_XS (15.48 GB) UkisAI IQ4_XS (15.48 GB) UkisAI Q4_K_M (17.44 GB)
German prose 0.01557 ± 0.00034 0.01714 0.01567 0.01556 0.01262
English prose 0.01502 ± 0.00041 0.01665 0.01437 0.01448 0.01320
German technical docs 0.01894 ± 0.00033 0.02110 0.02001 0.01995 0.01557
English technical docs 0.02082 ± 0.00068 0.02100 0.02056 0.02062 0.01634
code 0.01114 ± 0.00023 0.01253 0.01114 0.01125 0.00890
German dialogue 0.25136 ± 0.00795 0.26775 0.25494 0.25519 0.21366
math 0.04268 ± 0.00275 0.04729 0.04195 0.04038 0.03674
agent traces 1.42618 ± 0.02400 1.49850 1.44692 1.45086 1.39745
mean KLD 0.22521 0.23775 0.22820 0.22854 0.21431
mean top-1 agreement 92.70 % 92.34 % 92.66 % 92.66 % 93.12 %

Paired over sections against the plain IQ4_XS Kate file: better on 6 of 8 kinds, equal within error on English
technical docs and math, worse on none; top-1 agreement up on every kind; mean KLD −5.3 %. Against their own BF16,
Kate is level with bartowski's and UkisAI's IQ4_XS of the parent model (different references, so read it as parity).
Caveat: llama-perplexity in my fork does not parse special tokens, so the chat kinds (dialogue, math, agent) are
tokenized literally. That is the same for every file in the table, but not how the model sees them in use; it is also
why their absolute numbers are high.

Katy and Kate on the same tasks, same build:

Katy Kate
KLD E / W / K (Python reference, German Wikipedia, German Kubernetes docs; against own BF16) 0.0647 / 0.0128 / 0.0160 0.0612 / 0.0112 / 0.0141
KLD E / W / K against Katy's BF16 (the price of abliteration plus quant) 0.0647 / 0.0128 / 0.0160 0.0661 / 0.0116 / 0.0154
ARC-Challenge (1172) 94.3 % (92.8–95.5) 94.2 % (92.7–95.4)
GSM8K (250) 97.2 % (94.3–98.6) 97.6 % (94.9–98.9)
LiveCodeBench (100), no thinking / thinking budget 8000 42 / 67 44 / 65
Agent tasks (24, 8k–32k prompts): tool use / text 0.393 / 0.399 0.446 / 0.429
Structured output: JSON (40), long output (9), tool calls (40), agent loops (10) 40 / 5 / 38 / 9 39 / 6 / 38 / 9
Loops (responses stuck in repetition, all outputs) 7 6

Speed

One RTX PRO 4500 Blackwell (32 GB), FLE, DSpine head with n-max 15 plus n-gram drafting, f16 KV cache, paired A-B-B-A.
Kate against Katy (Katy with its own trained head), paired: agent tasks 152.6 against 161.5 tok/s, −5.5 % (95 %: −18.2 to
+5.5, unclear); code (LiveCodeBench) 153.6 against 152.3 tok/s, +0.9 % (−4.0 to +5.9, within noise). For reference, Katy in real
agent use: median 154.6 tok/s over 1,729 responses.

Smallest card (memory limit simulated with -fitt, same build): 20 GB keeps all 66 layers on the GPU at full speed
(142.7 tok/s); 18 GB 111.9; 16 GB 37.3; 14 GB 22.8; 12 GB 14.1 tok/s.

Start command I use (FLE; upstream llama.cpp needs a DSpine-capable build):

llama-server -m kate-27b-IQ4_XS-imat-R1-A.gguf -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf --spec-type draft-dflash,ngram-mod \
  --spec-draft-n-max 15 --spec-draft-p-min 0 --spec-draft-ngl all -ngl 99 -c 163840 -fa on -ctk f16 -ctv f16 \
  --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 32 --spec-ngram-mod-n-match 16 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0

Full report

Method, every measurement with intervals, negative results (including a benchmark bug in my own harness that had
inflated paired runs by up to 68 %): the report as a video: https://youtu.be/Bye6WPpkxA4 (the written version will be linked here).

Licence

Swift Open License v1.0 for the Swift contribution (see LICENSE, including the commercial-use threshold in §5),
Apache-2.0 for the Qwen3.8-27B base and the DSpine head (see LICENSE-APACHE-2.0), details in NOTICE.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-16 Card: banner, FLE Link 0.1.16 released 2c025f1 10.2 KB
    Loading...
  2. 2026-11-16 Add README.md 2c25fa9 10.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration