license: apache-2.0
base_model:
- KRAFTON/Raon-1-nano
base_model_relation: finetune
pipeline_tag: text-generation
library_name: gguf
tags: - gguf
- llama.cpp
- abliterated
- heretic
- raon
Raon-1-nano-Abliterated — GGUF
An abliterated build of KRAFTON/Raon-1-nano (dense 3.1B,
a thinking model on a 4-stream mHC residual with 18 RDA linear-attention and 6 NoPE MLA layers). It was
produced with Heretic 2.1 (its default Arbitrary-Rank Ablation method)
and quantized with KRAFTON's llama.cpp fork. Every claim below is measured against KRAFTON's own Q8_0.
Modification notice. These weights are a modified derivative. Ablation altered 20 tensors to
reduce refusals (see What changed). Nothing else was retrained.
KRAFTON did not produce, endorse, or review this build, and its behavior is not theirs.
Read this first
This is an escape hatch for over-refusal, not an uncensored model. Abliteration removes a refusal
direction; it adds no knowledge. This build knows what stock Raon-1-nano knows and nothing more. What
changed is its willingness to say it. If stock answers your prompt, use stock.
It needs KRAFTON's llama.cpp fork. Upstream llama.cpp does not support the raon1nano architecture.
Build the raon-1-nano-preview
branch (see Usage); these files were converted and tested at commit 4d72ff0.
It softens rather than refuses. It refused none of the test prompts, but about a quarter of its
answers quietly substitute a safer task or stay vague. Expect that on bare prompts.
The hazard worth naming: refusals double as hedging. Remove them and you get a model answering
fluently in domains where it is unreliable, with none of the cues that used to mark the boundary. At
3.1B the answers are often generic outlines rather than expert detail. You are accountable for what you
generate and for applicable law.
Measurements
All on Q8_0, against KRAFTON's published Q8_0 (sha256 5b4d73ea…6e146d22421). The files differ only
in the general.name metadata field and in the ablated tensors: every untouched tensor sampled by ranged
reads (RDA, MLA, mHC and MLP tensors from layers 0, 3, 11, 22 and 23, outside the ablated range) was
byte-identical, and both ablated tensors sampled from layer 15 differ, so it serves as the stock arm. Sampling is temp 1.0 / top-p 0.95 / top-k 20, seeded per item so both builds see the same
randomness. Refusals use mlabonne/harmful_behaviorstest[60:], the 44 prompts Heretic's optimiser never saw (it scored test[:60]), with Heretic's default
system prompt. Budgets: 1536 tokens thinking off, 8192 thinking on.
Refusals, sorted by hand. Heretic's English marker list flagged 3 (thinking off) and 4 (thinking on)
of the abliterated answers; reading them, every one was a compliance with disclaimer wording inside. It
also missed three of stock's refusals, which Raon wrote in Korean ("죄송하지만, 그 요청은 도와드릴 수
없습니다"). So both columns are read, not counted. No abliterated answer contained Korean.
| regime | stock refused | abliterated: refused / complied / deflected / soft |
|---|---|---|
enable_thinking: false |
44/44 | 0 / ~32 / ~7 / ~5 |
enable_thinking: true |
44/44 | 0 / ~32 / ~5 / ~7 |
- Deflected = answers a different, harmless version of the request. Soft = on topic but generic
or sandboxed ("simulated", "for illustration"). The sort is by hand from each answer's opening and is
approximate (±2 per cell). - Length. Answers average 341 tokens thinking off and 357 thinking on (stock's refusals: 89 and 225).
None reached its budget. - Thinking on, it answers these prompts without reasoning. On 42 of 44, the reasoning block is
empty; stock reasons for ~1k characters before refusing. It still reasons on math (below), so this is
specific to these requests, not a disabled thinking mode. - Repetition. 1 of 88 abliterated answers has more than 10% repeated 20-grams.
GSM8K, paired, n=100, enable_thinking: true:
| correct | mean tokens | truncated (8192 tokens) | |
|---|---|---|---|
| stock | 93/100 | 700 | 0 |
| abliterated | 93/100 | 690 | 1 |
Discordant pairs split 1/1 (McNemar exact p = 1.000): no math regression is detectable.
Abliteration damage tends to land in math reasoning, which is why this is the check that matters.
n=100 rules out a large regression, not a 2–3 pp one.
Unmeasured: MMLU, Korean-language tasks, coding, long context, agentic/tool-calling loops.
Files
| file | size |
|---|---|
Raon-1-nano-Abliterated-Q4_K_M.gguf |
1.94 GiB |
Raon-1-nano-Abliterated-Q6_K.gguf |
2.46 GiB |
Raon-1-nano-Abliterated-Q8_0.gguf |
3.16 GiB |
The same quant set KRAFTON publishes, all from one F16 GGUF converted with --no-mtp, without an
imatrix. The fork's llama-quantize keeps some Raon tensors at higher precision on its own. There is no
MTP variant: KRAFTON's MTP head was trained against the stock trunk. Only Q8_0 was evaluated. build/
holds the search record, not model files: the log (run.log), the Optuna study journal (all 100
trials), the Heretic config (heretic.toml), the list of merged tensors (heretic_deltas.json), the
stock-conversion comparison (stock-check.txt) and SHA256SUMS.
The GGUF metadata field general.name reads Abliterated (the converter took it from the build
directory's name). It is cosmetic; the tensors and chat template are what is described here.
Usage
Build KRAFTON's fork (Metal is on by default on Apple silicon):
git clone -b raon-1-nano-preview https://github.com/krafton-ai/llama.cpp
cd llama.cpp
cmake -B build && cmake --build build --config Release -j --target llama-server
Serve with KRAFTON's recommended "General" sampling (for agentic or code work they recommend temp 1.0,
presence penalty 0):
./build/bin/llama-server -m Raon-1-nano-Abliterated-Q8_0.gguf \
-ngl 99 -c 65536 --jinja \
--reasoning on --chat-template-kwargs '{"enable_thinking": true}' \
--temp 0.1 --presence-penalty 1.5
KRAFTON designed Raon-1-nano for thinking mode. Per request,"chat_template_kwargs": {"enable_thinking": false} turns it off. Only the 6 MLA layers keep a KV cache
(~7 KiB/token), so long contexts are cheap.
What changed
The merged bf16 weights differ from the original in exactly these tensors:
| tensor type | changed | layers |
|---|---|---|
attention output o_proj (attn_output), RDA and MLA layers |
10 | 12–21 |
MLP down_proj (ffn_down) |
10 | 12–21 |
| everything else (embeddings, per-layer embeddings, mHC, RDA/MLA projections, norms) | 0 | — |
Layers 0–11 and 22–23 are untouched entirely. KRAFTON's converter folds the mHC write-back factor intoo_proj and down_proj; it runs after the merge, so it applies to the abliterated tensors the same way.
Reproducing
Raon-1-nano's remote code is written for transformers 5 and needs no model patch under Heretic 2.1: it
returns hidden states already averaged over the 4 residual streams, Heretic finds o_proj on both layer
kinds plus down_proj (48 modules), and it keeps padded positions out of the RDA recurrence itself. This
build loaded a small in-process patch that does two things only:
- Renders prompts with
enable_thinking=False. The template's think-off prompt ends<think></think>\n\n, which a</think>response prefix would not reproduce byte for byte. - Writes the merge onto the original bf16 shards. This keeps the separate tied-head file and the
fp32 parameters exactly as shipped.
Before the search, a tiny random model checked hidden states, causality, padded-vs-unpadded and
cached-vs-uncached generation, and the merged tensor names. On the real weights, padded-vs-unpadded KL
was ≤ 0.0042 in bf16 on three of four prompts and 0.0112 on the fourth, a near-tied first token. All
four were 0.00000 in fp32, so the bf16 gap is rounding, not a padding leak. The search is stock Heretic
2.1.0 (transformers 5.22.2, peft 0.21.2), with this config:
response_prefix = ""
modifiers = [ { plugin = "heretic.modifiers.ara.ARA" } ]
[scorer.KeywordRate.prompts]
dataset = "mlabonne/harmful_behaviors"
split = "test[:60]"
column = "text"
[scorer.KLDivergence.prompts]
dataset = "mlabonne/harmless_alpaca"
split = "test[:60]"
column = "text"
and --seed 1234 --max-batch-size 128 --max-response-length 64 --n-trials 100 --n-startup-trials 40.
Abliteration is not bit-reproducible (the TPE search lands on a different trial each run). The
selected trial was 83: 1/60 refusals @ KL 0.0543, from a baseline of 59/60. Its parameters, as
Heretic reports them:
start_layer_index = 12
end_layer_index = 22
preserve_good_behavior_weight = 0.9613
steer_bad_behavior_weight = 0.1297
overcorrect_relative_weight = 0.8912
neighbor_count = 6
Verifying
60bc9c2181a3d87a8f93ddad4042aa48458627f2f6bd9631b113edbadf3e1027 Raon-1-nano-Abliterated-Q4_K_M.gguf
b96af0c70a58d5d97e6d0d0b0c5af3fd6da99554b26b927a377c6fbce50e9188 Raon-1-nano-Abliterated-Q6_K.gguf
a04987afbec975a35bb52fad3623455d98fb3218b42fd3ea8ad0c65ac72db1aa Raon-1-nano-Abliterated-Q8_0.gguf
License
Apache-2.0, inherited from KRAFTON/Raon-1-nano. The LICENSE here is the base repo's file, copied
verbatim (sha256 cfc7749b…, identical to upstream); it is the standard Apache License 2.0 text with no
added terms. No NOTICE file exists upstream, so none is propagated. As required by §4(b), this card
states that the files were modified (see the notice at the top). Abliteration and quantization change
nothing about the license.
Credits
- KRAFTON: the Raon-1-nano base model and the llama.cpp fork that runs it
- p-e-w/heretic: the abliteration method and tooling
- ggml-org/llama.cpp: the inference engine the fork builds on
- mlabonne: the prompt sets used for directions and scoring
Supporting my work
If these quants are useful to you, consider supporting them via Ko-fi. It goes toward local hardware for quantizing and testing larger models. I also build Reika, a coding agent CLI designed around small local models first.