← back to catalog · registered 2026-11-12 06:03

hUh63/Kestrel-4B-Coder-Decensored-MTP-Q4_K_M-GGUF

hUh63 4B GGUF multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/hUh63%2FKestrel-4B-Coder-Decensored-MTP-Q4_K_M-GGUF"
Response includes
  • classification m8
  • files 9
  • author_summary 7 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
629
Likes
0
Model age
1w ago
created 2026-11-12
Downloads over time
Now 629 → from 0 ↑ 0%
0231461692 0 on Nov 12 629 on Nov 22 Nov
Nov 12 → Nov 22 · 11 snapshots · spans 10 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ko ja ru es fr
Quantizations
Q4_K
Tags
gguf q4_k_m qwen3_5 coder code-generation mtp speculative-decoding decensored heretic abliteration image-text-to-text base_model:Jackrong/Kestrel-4B-Coder

Related

Total size
2.61 GB
Files
9
Quantizations
3
Registered
2026-11-12 06:03
Last updated on HF
2026-11-18 12:29

Files by quantization

Q4_K 1 file 2.61 GB
Kestrel-4B-Coder-Decensored-MTP-Q4_K_M.gguf 2.61 GB bd67bcbb download
F16 1 file 641 MB
mmproj-F16.gguf 641 MB ee834f57 download
Auxiliary files 7 files 799 KB
calibration.txt 761 KB 7a070587 download
README.md 12.1 KB 392e7eda download
LICENSE 11.1 KB 1d5180a4 download
QUANTIZATION.md 10.4 KB c389d8ef download
ablation_report.json 3.35 KB e391848f download
.gitattributes 1.61 KB eb105867 download
tensor_types.txt 18.0 B 79d41c75 download

README current version from Hugging Face


library_name: transformers
base_model: Jackrong/Kestrel-4B-Coder
base_model_relation: finetune
license: apache-2.0
license_link: https://huggingface.co/Jackrong/Kestrel-4B-Coder/blob/main/LICENSE
quantized_by: hUh63
language:

  • en
  • zh
  • ko
  • ja
  • ru
  • es
  • fr
    pipeline_tag: image-text-to-text
    tags:
  • qwen3_5
  • unsloth
  • coder
  • code-generation
  • tool-use
  • function-calling
  • agent
  • mtp
  • multi-token-prediction
  • speculative-decoding
  • gguf
  • llama.cpp
  • q4_k_m
  • imatrix
  • decensored
  • uncensored
  • abliterated
  • vision
  • multimodal
  • conversational

Kestrel-4B-Coder — Decensored · MTP · Q4_K_M GGUF

This is an unofficial community derivative. All model weights originate from
Jackrong/Kestrel-4B-Coder by Jackrong,
itself a fine-tune of Qwen3.5-4B by the Qwen team
(prepared from unsloth/Qwen3.5-4B).
This repository is not affiliated with, endorsed by, or validated by Jackrong, Qwen,
Alibaba, or Unsloth. See Attribution below.

A single-file Q4_K_M GGUF of the Kestrel-4B-Coder coding model with three properties:

  1. Decensored — refusal behaviour was reduced with a training-free, direction-level
    ablation (see What was actually done). No fine-tuning, no
    retraining, no weight interpolation.
  2. Built-in MTP head — the original mtp.* tensors are bundled into the same GGUF as
    layer 33 (blk.32.nextn.*). Use --spec-type draft-mtp with one file; no second
    draft model is needed.
  3. Quality-preserving quantization — llama.cpp Q4_K_M with an importance matrix
    computed from ~760 KiB of real multilingual / math / code / reasoning / agent text,
    and the entire MTP/NextN layer held at Q6_K to keep speculative-decoding acceptance
    rate stable.

file_type = 15 (MOSTLY_Q4_K_M). The visual projector is a separate mmproj-F16.gguf
(normal for a multimodal model); it is optional for text-only coding.

Contents

File What it is
Kestrel-4B-Coder-Decensored-MTP-Q4_K_M.gguf 2,807,777,984 B (2.61 GiB) — text backbone + tokenizer + built-in MTP head, Q4_K_M
mmproj-F16.gguf 672,423,776 B — F16 vision projector (optional for text-only use)
calibration.txt The exact imatrix corpus used
tensor_types.txt The exact per-tensor precision override used at quantize time
ablation_report.json Full de-censoring measurements (baseline vs. final, all candidates)
QUANTIZATION.md Reproducible recipe (environment, commands, pitfalls)
LICENSE Apache-2.0, inherited from the source model

Download only what you need (do not clone the whole repo if you only want the model):

huggingface-cli download hUh63/Kestrel-4B-Coder-Decensored-MTP-Q4_K_M-GGUF \
  Kestrel-4B-Coder-Decensored-MTP-Q4_K_M.gguf --local-dir .

What was actually done

1. De-censoring (real, measured, weight-level)

We followed the method used by Heretic / Arditi et al.
(2024), "Refusal in Language Models Is Mediated by a Single Direction", adapted to the
Qwen3.5 hybrid architecture (24 Gated DeltaNet linear_attention layers + 8 full
self_attention layers):

  • Real probes: 48 harmless prompts from mlabonne/harmless_alpaca and 48 harmful
    prompts from mlabonne/harmful_behaviors, rendered through the model's own chat
    template (You are a helpful assistant.).
  • Real forward passes on CPU with transformers 5.20.2 (pure-PyTorch Gated DeltaNet
    fallback), collecting the residual-stream hidden state at the last prompt token for
    every layer.
  • Per-layer refusal direction r_l = normalize(mean_harmful_l - mean_harmless_l),
    then projected orthogonalization against the harmless direction (grimjim / MPOA)
    so that only the component orthogonal to benign representations is removed.
  • Rank-1 ablation on the matrices that write into the residual stream only:
    self_attn.o_proj (full-attn layers), linear_attn.out_proj (Gated DeltaNet layers),
    and mlp.down_proj. W' = W − s·r_l(r_lᵀW), followed by row-magnitude preservation
    (Lai 2025, MPOA) to limit capability loss. Ablation strength peaks in the middle/late
    layers and tapers toward both ends, mirroring Heretic's weight-distribution parameter.
  • Candidate grid with hard gates: 6 candidates, each really applied to the weights and
    re-evaluated. Selected by (a) drop in harmful-vs-harmless separability on the refusal
    direction, with (b) a hard constraint that harmless residual representations stay
    cosine-similar to the original. Exact numbers are in ablation_report.json.

Measured on this model (12 held-out prompts per class, layers 12–27):

baseline final
harmful projection on refusal direction 7.3356 0.7373
harmless projection -0.0011 -0.0446
separability (harmful − harmless) 7.3367 0.7819 (−89.3%)
harmless-representation cosine vs. original 1.0000 0.9950

Measured on 12 held-out prompts per class, layers 12–27, with the selected
configuration strength=1.2, attn_w=1.0, mlp_w=0.5. Two stronger candidates that
removed more of the refusal direction were rejected because the harmless cosine
fell below the 0.995 hard gate (0.9919 / 0.9875). All six candidates and their
measurements are in ablation_report.json. We do not claim any number that was not
measured.

Untouched: all mtp.* tensors (the MTP head), all model.visual.* tensors (vision
tower), the embedding matrix, every LayerNorm / RMSNorm, and every projection other than
the three output projections listed above. De-censoring a coding model is not a license
to break its speculative-decoding head.

2. Quantization (real, environment-matched)

Reproduced on 8 CPU cores / 16 GiB RAM / no GPU / Debian 12 / Python 3.11:
llama.cpp (commit 5d4c8ee, CPU build), torch 2.11.0+cpu, transformers 5.20.2
(isolated), gguf. Full commands are in QUANTIZATION.md.

  • BF16 GGUF conversion with the mtp.* tensors in the same file →
    qwen35.block_count = 33, qwen35.nextn_predict_layers = 1, 15 blk.32.* tensors.
  • llama-imatrix with 16 chunks of 256 tokens, --nextn so the MTP layer
    participates in importance statistics (measured: imatrix.n_layer_nextn=1, 1024
    statistics entries including blk.32.*).
  • llama-quantize ... Q4_K_M with --imatrix and a tensor-type override
    ^blk\.32\..*=Q6_K (anchored regex; the default output.weight → Q6_K rule is kept,
    attn_v / some ffn_down are Q6_K by Q4_K_M's own mixed rule). Holding the draft head
    at higher precision is the community-tested way to keep MTP acceptance rate from
    collapsing after quantization; it costs only ~30 MiB.

Speculative decoding is lossless; quantization is lossy. These are different things:
--spec-type draft-mtp proposes candidates that the backbone still verifies token by
token (rejected drafts are discarded, sampling distribution unchanged), while Q4_K_M
inevitably loses precision relative to the BF16 source. The MTP head here came from the
original aligned checkpoint and only proposes tokens; every accepted token is still
produced by the de-censored backbone.

Usage

Requires a recent llama.cpp build with Qwen3.5
and draft-mtp support.

huggingface-cli download hUh63/Kestrel-4B-Coder-Decensored-MTP-Q4_K_M-GGUF \
  Kestrel-4B-Coder-Decensored-MTP-Q4_K_M.gguf --local-dir .

llama-cli -m Kestrel-4B-Coder-Decensored-MTP-Q4_K_M.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  -p "Write a Python function to merge two sorted lists." \
  -n 256 -c 4096 -t 8 --jinja

Do not pass -md / a second model: the drafter is blk.32.nextn.* inside this file.
Startup logs should show blk.32.nextn.eh_proj.weight and draft-mtp active.

Measured on 8 CPU cores, 128-token generation, one prompt, this exact file:

Run Generation speed Draft acceptance
no speculation 9.2 tok/s —
--spec-type draft-mtp --spec-draft-n-max 2, one file 9.8 tok/s 71 / 112 (63.4%)

Trace log confirms adding speculative implementation 'draft-mtp',
called impl draft-mtp, gen = 2, set_embeddings_nextn: value = 1 and
loading tensor blk.32.nextn.eh_proj.weight. The modest speedup is expected for a
4B model on 8 CPU cores — drafting has its own cost. The important properties are
lossless, single-file, no external draft model. Acceptance rate is
workload-dependent; this is one measurement, not a benchmark.

For image input, add --mmproj mmproj-F16.gguf with a multimodal-capable binary
(llama-mtmd-cli / llama-server). Vision quality was not evaluated for this release.

To reproduce the whole pipeline (de-censoring, conversion, imatrix, quantize, upload),
see QUANTIZATION.md and the method documents in the CNB repository that produced it.

Limitations & disclaimer

  • This is an uncensored / decensored community experiment. Safety alignment was
    deliberately reduced. Outputs may be unsafe, biased, or wrong. You are solely
    responsible for lawful, compliant use and for reviewing generated code before running
    it. This is not suitable for unsupervised or safety-critical deployment.
  • Quantization is lossy. Q4_K_M is a ~4.5 BPW approximation of the BF16 source.
    Coding/agent quality will differ from the author's Q8_0 results. We did not run
    HumanEval / HumanEval+ / HermesAgent-20 on this quantized derivative; the scores in
    the original model card belong to
    the original high-precision model and are not results for this file.
  • No benchmark, no "quality preserved" claim beyond what is actually measured in
    ablation_report.json and check_gguf.py output. We did not run HumanEval /
    HermesAgent-20 on this quantized derivative. MTP acceptance rate varies with prompt
    style, context length and sampling settings.
  • imatrix corpus is limited in size; multilingual coverage is en/zh/fr/es/ja/ko/ru.
  • The de-censoring was computed on CPU with a small probe set. It reduces the measured
    refusal direction; it does not guarantee behaviour on every prompt.

Attribution / license

The weights are not our work. Full provenance, kept intact:

  1. Qwen team, Alibaba Cloud — Qwen3.5-4B,
    the foundation model family.
  2. Unsloth — unsloth/Qwen3.5-4B and the
    open-source Unsloth fine-tuning framework.
  3. Jackrong — Kestrel-4B-Coder,
    the ~150K-example coding/agent fine-tune (distilled from Qwen3.8-27B,
    Qwen3.8-Flash-Next, Qwen3.8-Max, nvidia/Nemotron). Direct source of every weight
    in this repository
    , including the MTP head and the vision tower.
    Thanks also to Kyle Hessling (collaboration/testing/hardware), NVIDIA (Nemotron),
    EvalPlus, llama.cpp, LM Studio and BenchLocal / Steven Lei as credited
    in the original card.
  4. Philipp Emanuel Weidmann (p-e-w) — Heretic,
    the open-source de-censoring method this derivative follows (AGPL-3.0). Used as a
    reference implementation; no Heretic code is distributed here.
  5. mlabonne — the harmless_alpaca / harmful_behaviors probe datasets.
  6. hUh63 — de-censoring application, GGUF conversion, imatrix, Q4_K_M quantization
    and publication only.

License: Apache-2.0, inherited from the source model. A copy of the source LICENSE is
included. The conversion/quantization scripts are provided under MIT in the CNB method
repository. This derivative is offered under the same terms as the source; if you are a
rights holder and want it removed, please open an issue.

@misc{jackrong2026kestrel4bcoder,
  title        = {Kestrel-4B-Coder},
  author       = {Jackrong},
  year         = {2026},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/Jackrong/Kestrel-4B-Coder}
}

README history 10 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-18 Update README.md 3ed154b 6.1 KB
    Loading...
  2. 2026-11-18 Update README.md af1d9fa 6.1 KB
    Loading...
  3. 2026-11-15 Add Heretic 2.1.0.dev0 Abliteration cheapest-point model card and reproducibi... 269c334 6.1 KB
    Loading...
  4. 2026-11-15 Add Heretic 2.1.0.dev0 Abliteration cheapest-point model card and reproducibi... 9417ada 6.1 KB
    Loading...
  5. 2026-11-12 Rewrite model card for Heretic 2.0.0 ARA build + add reproducible recipe and ... 39f754c 6.9 KB
    Loading...
  6. 2026-11-12 Rewrite model card for Heretic 2.0.0 ARA build + add reproducible recipe and ... c2b271f 6.9 KB
    Loading...
  7. 2026-11-12 docs: sync measured imatrix settings and runtime numbers with the actual run af1e29e 12.1 KB
    Loading...
  8. 2026-11-12 docs: sync measured imatrix settings and runtime numbers with the actual run 1c55d9d 12.1 KB
    Loading...
  9. 2026-11-12 Add decensored Kestrel-4B-Coder MTP Q4_K_M GGUF (imatrix + high-precision NextN) 6947c7c 12 KB
    Loading...
  10. 2026-11-12 Add decensored Kestrel-4B-Coder MTP Q4_K_M GGUF (imatrix + high-precision NextN) ca0b8a6 12 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration