← back to catalog · registered 2026-11-13 09:03

alexwkleung/Raon-1-nano-Abliterated-GGUF

Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/alexwkleung%2FRaon-1-nano-Abliterated-GGUF"
Response includes
  • classification m8
  • files 6
  • author_summary 9 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
557
↑ 4,542% in 90 days
Likes
1
Model age
1w ago
created 2026-11-11
Downloads over time
Now 557 → from 12 ↑ 4,542%
0204408612 12 on Nov 13 557 on Nov 22 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Quantizations
Q4_K Q6_K Q8_0
Tags
gguf llama.cpp abliterated heretic raon text-generation base_model:KRAFTON/Raon-1-nano base_model:finetune:KRAFTON/Raon-1-nano license:apache-2.0 endpoints_compatible region:us conversational

Related

Total size
7.56 GB
Files
6
Quantizations
4
Registered
2026-11-13 09:03
Last updated on HF
2026-11-13 08:13

Files by quantization

Q8_0 1 file 3.16 GB
Raon-1-nano-Abliterated-Q8_0.gguf 3.16 GB a04987af download
Q6_K 1 file 2.46 GB
Raon-1-nano-Abliterated-Q6_K.gguf 2.46 GB b96af0c7 download
Q4_K 1 file 1.94 GB
Raon-1-nano-Abliterated-Q4_K_M.gguf 1.94 GB 60bc9c21 download
Auxiliary files 3 files 23.8 KB
LICENSE 11.1 KB d6456956 download
README.md 11.0 KB 24118828 download
.gitattributes 1.69 KB 3f36b3cf download

README current version from Hugging Face


license: apache-2.0
base_model:

  • KRAFTON/Raon-1-nano
    base_model_relation: finetune
    pipeline_tag: text-generation
    library_name: gguf
    tags:
  • gguf
  • llama.cpp
  • abliterated
  • heretic
  • raon

Raon-1-nano-Abliterated — GGUF

An abliterated build of KRAFTON/Raon-1-nano (dense 3.1B,
a thinking model on a 4-stream mHC residual with 18 RDA linear-attention and 6 NoPE MLA layers). It was
produced with Heretic 2.1 (its default Arbitrary-Rank Ablation method)
and quantized with KRAFTON's llama.cpp fork. Every claim below is measured against KRAFTON's own Q8_0.

Modification notice. These weights are a modified derivative. Ablation altered 20 tensors to
reduce refusals (see What changed). Nothing else was retrained.
KRAFTON did not produce, endorse, or review this build, and its behavior is not theirs.

Read this first

This is an escape hatch for over-refusal, not an uncensored model. Abliteration removes a refusal
direction; it adds no knowledge. This build knows what stock Raon-1-nano knows and nothing more. What
changed is its willingness to say it. If stock answers your prompt, use stock.

It needs KRAFTON's llama.cpp fork. Upstream llama.cpp does not support the raon1nano architecture.
Build the raon-1-nano-preview
branch (see Usage); these files were converted and tested at commit 4d72ff0.

It softens rather than refuses. It refused none of the test prompts, but about a quarter of its
answers quietly substitute a safer task or stay vague. Expect that on bare prompts.

The hazard worth naming: refusals double as hedging. Remove them and you get a model answering
fluently in domains where it is unreliable, with none of the cues that used to mark the boundary. At
3.1B the answers are often generic outlines rather than expert detail. You are accountable for what you
generate and for applicable law.

Measurements

All on Q8_0, against KRAFTON's published Q8_0 (sha256 5b4d73ea…6e146d22421). The files differ only
in the general.name metadata field and in the ablated tensors: every untouched tensor sampled by ranged
reads (RDA, MLA, mHC and MLP tensors from layers 0, 3, 11, 22 and 23, outside the ablated range) was
byte-identical, and both ablated tensors sampled from layer 15 differ, so it serves as the stock arm. Sampling is temp 1.0 / top-p 0.95 / top-k 20, seeded per item so both builds see the same
randomness. Refusals use mlabonne/harmful_behaviors
test[60:], the 44 prompts Heretic's optimiser never saw (it scored test[:60]), with Heretic's default
system prompt. Budgets: 1536 tokens thinking off, 8192 thinking on.

Refusals, sorted by hand. Heretic's English marker list flagged 3 (thinking off) and 4 (thinking on)
of the abliterated answers; reading them, every one was a compliance with disclaimer wording inside. It
also missed three of stock's refusals, which Raon wrote in Korean ("죄송하지만, 그 요청은 도와드릴 수
없습니다"). So both columns are read, not counted. No abliterated answer contained Korean.

regime stock refused abliterated: refused / complied / deflected / soft
enable_thinking: false 44/44 0 / ~32 / ~7 / ~5
enable_thinking: true 44/44 0 / ~32 / ~5 / ~7
  • Deflected = answers a different, harmless version of the request. Soft = on topic but generic
    or sandboxed ("simulated", "for illustration"). The sort is by hand from each answer's opening and is
    approximate (±2 per cell).
  • Length. Answers average 341 tokens thinking off and 357 thinking on (stock's refusals: 89 and 225).
    None reached its budget.
  • Thinking on, it answers these prompts without reasoning. On 42 of 44, the reasoning block is
    empty; stock reasons for ~1k characters before refusing. It still reasons on math (below), so this is
    specific to these requests, not a disabled thinking mode.
  • Repetition. 1 of 88 abliterated answers has more than 10% repeated 20-grams.

GSM8K, paired, n=100, enable_thinking: true:

correct mean tokens truncated (8192 tokens)
stock 93/100 700 0
abliterated 93/100 690 1

Discordant pairs split 1/1 (McNemar exact p = 1.000): no math regression is detectable.
Abliteration damage tends to land in math reasoning, which is why this is the check that matters.
n=100 rules out a large regression, not a 2–3 pp one.

Unmeasured: MMLU, Korean-language tasks, coding, long context, agentic/tool-calling loops.

Files

file size
Raon-1-nano-Abliterated-Q4_K_M.gguf 1.94 GiB
Raon-1-nano-Abliterated-Q6_K.gguf 2.46 GiB
Raon-1-nano-Abliterated-Q8_0.gguf 3.16 GiB

The same quant set KRAFTON publishes, all from one F16 GGUF converted with --no-mtp, without an
imatrix. The fork's llama-quantize keeps some Raon tensors at higher precision on its own. There is no
MTP variant: KRAFTON's MTP head was trained against the stock trunk. Only Q8_0 was evaluated. build/
holds the search record, not model files: the log (run.log), the Optuna study journal (all 100
trials), the Heretic config (heretic.toml), the list of merged tensors (heretic_deltas.json), the
stock-conversion comparison (stock-check.txt) and SHA256SUMS.

The GGUF metadata field general.name reads Abliterated (the converter took it from the build
directory's name). It is cosmetic; the tensors and chat template are what is described here.

Usage

Build KRAFTON's fork (Metal is on by default on Apple silicon):

git clone -b raon-1-nano-preview https://github.com/krafton-ai/llama.cpp
cd llama.cpp
cmake -B build && cmake --build build --config Release -j --target llama-server

Serve with KRAFTON's recommended "General" sampling (for agentic or code work they recommend temp 1.0,
presence penalty 0):

./build/bin/llama-server -m Raon-1-nano-Abliterated-Q8_0.gguf \
  -ngl 99 -c 65536 --jinja \
  --reasoning on --chat-template-kwargs '{"enable_thinking": true}' \
  --temp 0.1 --presence-penalty 1.5

KRAFTON designed Raon-1-nano for thinking mode. Per request,
"chat_template_kwargs": {"enable_thinking": false} turns it off. Only the 6 MLA layers keep a KV cache
(~7 KiB/token), so long contexts are cheap.

What changed

The merged bf16 weights differ from the original in exactly these tensors:

tensor type changed layers
attention output o_proj (attn_output), RDA and MLA layers 10 12–21
MLP down_proj (ffn_down) 10 12–21
everything else (embeddings, per-layer embeddings, mHC, RDA/MLA projections, norms) 0 —

Layers 0–11 and 22–23 are untouched entirely. KRAFTON's converter folds the mHC write-back factor into
o_proj and down_proj; it runs after the merge, so it applies to the abliterated tensors the same way.

Reproducing

Raon-1-nano's remote code is written for transformers 5 and needs no model patch under Heretic 2.1: it
returns hidden states already averaged over the 4 residual streams, Heretic finds o_proj on both layer
kinds plus down_proj (48 modules), and it keeps padded positions out of the RDA recurrence itself. This
build loaded a small in-process patch that does two things only:

  • Renders prompts with enable_thinking=False. The template's think-off prompt ends
    <think></think>\n\n, which a </think> response prefix would not reproduce byte for byte.
  • Writes the merge onto the original bf16 shards. This keeps the separate tied-head file and the
    fp32 parameters exactly as shipped.

Before the search, a tiny random model checked hidden states, causality, padded-vs-unpadded and
cached-vs-uncached generation, and the merged tensor names. On the real weights, padded-vs-unpadded KL
was ≤ 0.0042 in bf16 on three of four prompts and 0.0112 on the fourth, a near-tied first token. All
four were 0.00000 in fp32, so the bf16 gap is rounding, not a padding leak. The search is stock Heretic
2.1.0 (transformers 5.22.2, peft 0.21.2), with this config:

response_prefix = ""
modifiers = [ { plugin = "heretic.modifiers.ara.ARA" } ]

[scorer.KeywordRate.prompts]
dataset = "mlabonne/harmful_behaviors"
split = "test[:60]"
column = "text"

[scorer.KLDivergence.prompts]
dataset = "mlabonne/harmless_alpaca"
split = "test[:60]"
column = "text"

and --seed 1234 --max-batch-size 128 --max-response-length 64 --n-trials 100 --n-startup-trials 40.

Abliteration is not bit-reproducible (the TPE search lands on a different trial each run). The
selected trial was 83: 1/60 refusals @ KL 0.0543, from a baseline of 59/60. Its parameters, as
Heretic reports them:

start_layer_index             = 12
end_layer_index               = 22
preserve_good_behavior_weight = 0.9613
steer_bad_behavior_weight     = 0.1297
overcorrect_relative_weight   = 0.8912
neighbor_count                = 6

Verifying

60bc9c2181a3d87a8f93ddad4042aa48458627f2f6bd9631b113edbadf3e1027  Raon-1-nano-Abliterated-Q4_K_M.gguf
b96af0c70a58d5d97e6d0d0b0c5af3fd6da99554b26b927a377c6fbce50e9188  Raon-1-nano-Abliterated-Q6_K.gguf
a04987afbec975a35bb52fad3623455d98fb3218b42fd3ea8ad0c65ac72db1aa  Raon-1-nano-Abliterated-Q8_0.gguf

License

Apache-2.0, inherited from KRAFTON/Raon-1-nano. The LICENSE here is the base repo's file, copied
verbatim (sha256 cfc7749b…, identical to upstream); it is the standard Apache License 2.0 text with no
added terms. No NOTICE file exists upstream, so none is propagated. As required by §4(b), this card
states that the files were modified (see the notice at the top). Abliteration and quantization change
nothing about the license.

Credits

  • KRAFTON: the Raon-1-nano base model and the llama.cpp fork that runs it
  • p-e-w/heretic: the abliteration method and tooling
  • ggml-org/llama.cpp: the inference engine the fork builds on
  • mlabonne: the prompt sets used for directions and scoring

Supporting my work

If these quants are useful to you, consider supporting them via Ko-fi. It goes toward local hardware for quantizing and testing larger models. I also build Reika, a coding agent CLI designed around small local models first.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-13 card: drop examples from the softening note a9a2001 11 KB
    Loading...
  2. 2026-11-13 Add model card (eval 2026-11-12) 7471874 11.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration