license: apache-2.0
base_model: ricursiveAI/Ripple-65B-Preview
language:

• en
  tags:
• abliterated
• ripple
• ricursive
• gguf
• llama.cpp
• reasoning
• tool-calling
  pipeline_tag: text-generation

Ripple-65B-Preview abliterated GGUF

Follow @procrastiness on Twitter/X for more model releases and updates.

A weight-edited, quantized build of ricursiveAI/Ripple-65B-Preview, the 64.78-billion-parameter dense model from Ricursive Intelligence, focused on agentic coding. All 80 layers are kept. A refusal direction is projected out of the token embeddings and 63 layers, and nothing is retrained or fine-tuned. Released under Apache-2.0 with downloadable weights and no hosted API.

This model answers many requests the original refuses, including dangerous ones. It has had no safety review. You are responsible for how you use it and for any safeguards if you expose it to other people.

The two columns differ in more than the edit. See Measured checks for the token budgets, prompt splits and importance matrix behind each number. The upstream card's benchmark scores belong to the unmodified BF16 model and were not re-run here, so don't read them as scores for these files.

Recommended starting setup

Start with Q4_K_M, a 64K context, one concurrent request and a generous output limit of 4,096 tokens or more. Ripple usually reasons before it answers, and that reasoning counts against your output limit.

Use the embedded chat template by passing --jinja, and pass --reasoning-format deepseek so reasoning arrives in its own field. The upstream repository publishes no recommended sampling settings, and sampling was not tuned for this release. Every measurement below uses temperature 0. For chat, start from llama-server's defaults.

Choose a quant and hardware

Download one quant. Q6_K and Q8_0 are split into two files, and llama.cpp loads the second part on its own when you point it at -00001-of-00002.gguf.

Memory and speed were measured on one machine: four NVIDIA V100s (one 32 GB, three 16 GB), --tensor-split 32,16,16,16, 65,536 total context across 8 slots. Your numbers will differ.

These are starting points rather than minimums. File size leaves out the context cache and runtime buffers.

1. Install llama.cpp and the download tool

Ripple has its own ricursive architecture, added to llama.cpp by PR #31692, merged 5 November 2026. Anything older fails to load these files. Packaged builds may not include it yet. This release was tested at commit 954de248ac6907dd05dc19f0f511ab18f2807a54. LM Studio, Ollama and other apps must bundle a backend that new, and none were tested.

Ubuntu / Debian: reproducible NVIDIA build

Install a compatible NVIDIA driver and the CUDA Toolkit first. nvidia-smi and nvcc --version should both work. Then:

sudo apt-get update
sudo apt-get install -y git cmake build-essential libcurl4-openssl-dev python3-venv
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 954de248ac6907dd05dc19f0f511ab18f2807a54
cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j 8 --target llama-server
export PATH="$PWD/build/bin:$PATH"
cd ..
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade huggingface_hub requests

For a CPU-only build, configure with -DGGML_CUDA=OFF and start the server with -ngl 0.

Windows

Install Python 3.10 or newer, then in PowerShell:

python -m pip install --upgrade huggingface_hub requests

Get a CUDA build dated 5 November 2026 or later from the llama.cpp releases, extract the whole archive, and keep its DLLs next to llama-server.exe. From that folder, replace llama-server in the commands below with .\llama-server.exe. Check the startup log to confirm it found your GPU.

macOS

brew install llama.cpp
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade huggingface_hub requests

If the Homebrew build is older than the PR above, build from source using the commands in the Ubuntu section, with -DGGML_CUDA=OFF.

2. Download the model

hf download 0xKitkat/Ripple-65B-Preview-abliterated-GGUF Ripple-65B-Preview-abliterated-Q4_K_M.gguf --local-dir ripple

For Q5_K_M, replace Q4_K_M in the filename. For a split quant, download both parts:

hf download 0xKitkat/Ripple-65B-Preview-abliterated-GGUF --include "*Q6_K-*" --local-dir ripple

Rerun the same command after an interruption. Checksums are in SHA256SUMS. On Linux, run sha256sum ripple/*.gguf; on Windows, Get-FileHash ripple\Ripple-65B-Preview-abliterated-Q4_K_M.gguf -Algorithm SHA256.

3. Start the local server

Automatic fitting

llama-server -m ripple/Ripple-65B-Preview-abliterated-Q4_K_M.gguf --alias ripple -c 65536 --parallel 1 --fit on --jinja --reasoning-format deepseek --host 127.0.0.1 --port 8080

Open http://127.0.0.1:8080 for the built-in chat UI. The API is at http://127.0.0.1:8080/v1 with the model name ripple. No API key is needed; if a client insists on one, use local. Keep the terminal open.

Several GPUs, all layers on GPU

llama-server -m ripple/Ripple-65B-Preview-abliterated-Q4_K_M.gguf --alias ripple -c 65536 --parallel 1 -ngl 999 --split-mode layer --tensor-split 32,16,16,16 --jinja --reasoning-format deepseek -b 1024 -ub 512 --host 127.0.0.1 --port 8080

Set --tensor-split to match your cards' memory. -ngl 999 means all layers; the model has 80. If the split goes to the wrong cards, set CUDA_DEVICE_ORDER=PCI_BUS_ID first, as nvidia-smi and CUDA can number them differently.

With --parallel N, the context is divided equally among the N slots. The test runs used -c 65536 --parallel 8, which gives each request 8,192 tokens.

4. Use the API from Python

Save as chat_ripple.py and run it while the server is up:

import requests

response = requests.post(
    "http://127.0.0.1:8080/v1/chat/completions",
    json={
        "model": "ripple",
        "messages": [{"role": "user", "content": "Explain how a rainbow forms in three sentences."}],
        "max_tokens": 4096,
        "stream": False,
    },
    timeout=3600,
)
response.raise_for_status()
choice = response.json()["choices"][0]
print("finish:", choice["finish_reason"])
print("reasoning:", choice["message"].get("reasoning_content"))
print("answer:", choice["message"]["content"])

For repeatable output, add "temperature": 0 and a "seed".

Tool calling

The embedded template renders OpenAI-style tools, and llama.cpp returns structured calls. A one-tool round trip passed on all four quants.

import json
import requests

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
response = requests.post(
    "http://127.0.0.1:8080/v1/chat/completions",
    json={
        "model": "ripple",
        "messages": [{"role": "user", "content": "What is the weather in Hanoi right now? Use the tool."}],
        "tools": tools,
        "max_tokens": 4096,
    },
    timeout=3600,
)
response.raise_for_status()
for call in response.json()["choices"][0]["message"].get("tool_calls", []):
    print(call["function"]["name"], json.loads(call["function"]["arguments"]))

Thinking and long context

There is no thinking switch. The chat template has no enable_thinking option and does not open <think> for the model; the model decides for itself. It opened with reasoning on 99.7% of harmful and 83% of harmless calibration prompts, but often skipped it on short prompts with strict formatting. In the "Think briefly" answer suites below, the median was 0 reasoning characters. To force reasoning, call /completion with a raw prompt ending in <|im_start|>assistant\n<think>\n, which is how the reasoning-mode checks ran.

Expect much longer reasoning on prompts the original refused. The reasoning rarely stops at a refusal now, so it carries on through the request. On the 48 test prompts the median reasoning rose from about 1,600 to about 7,600 characters. Even at 3,072 tokens, 11 of 48 hit the limit, 4 of them before any final answer. Zlib compression ratios gave no sign of repetition loops, though that is a coarse check. On harmless prompts, reasoning length barely changed (median about 1,400 characters both ways).

Leave reasoning_content out of the message history. If you send it back, the template includes it, and it uses up context.

Long context has had one test. The source advertises up to 1,048,576 tokens and the GGUF reports a 2,097,152-token training context. Here, Q4_K_M at -c 131072 --parallel 1 found a short fact placed a quarter of the way into a 54,657-token prompt. It processed the prompt at 329 tok/s and then decoded at 12 tok/s, using 47.2 GiB of GPU memory. Larger sizes were not tried. One needle test doesn't establish long-context quality.

Troubleshooting

Measured checks

All runs used llama.cpp commit 954de24, temperature 0, seed 20261107, 8 slots of 8,192 tokens and the four-V100 machine above. Each quant was loaded and tested on its own. Q6_K and Q8_0 ran these suites as whole files before splitting. Their split parts were then loaded from the first shard and passed a generation and speed check, with the same speed as the whole file.

How to read these:

• The refusal number is a regex count, and nothing more. It looks for refusal phrases in the first 800 characters of the final answer. It can miss a soft refusal, and it says nothing about whether an answer is accurate or useful. Prompts with no final answer count as neither refusal nor compliance.
• Nobody read the harmful-prompt outputs. Generated text was classified in memory and discarded, so only flags, lengths and SHA-256 hashes were stored. That is why this card makes no claim about the quality of those answers.
• The comparison is not perfectly matched. The original Q4_K_M has no importance matrix, while the released Q4, Q5 and Q6 do. The original's test-split run used a smaller budget, which is the most it needed, since its refusals finish early. Its false-refusal figure comes from the validation split with a 1,024-token budget.
• The earlier run with matched budgets agrees. The edited weights at Q4_K_M without an importance matrix scored 0/39 refused on the validation split, against 37/39 for the original with the same 1,024-token budget, and 34 of the 39 ran out of budget before answering. On 16 of those prompts at 4,096 tokens: 0/16 refused and 5 unfinished, with a median of 2,972 tokens.
• Small suites move by noise. One case flipping is not evidence of a real change. The quality loss between the original and Q4_K_M is one arithmetic case. These are functional checks for catching damage, not capability benchmarks, and they were reused during candidate checks, so they are not an independent holdout. The refusal prompts for the final numbers come from a test split not used for calibration or selection.
• Reasoning-mode results are the fairer check of normal use, since the other suites mostly skip reasoning. In both modes, the shared hard failure is one remainder puzzle that runs out of budget, and the shared code failure is one CSV-parsing task.

Full per-case results are in evaluations/, and the prompts are in build/eval_suite.json.

How the edit was made

Ripple usually reasons before answering, which changes what abliteration has to target. Almost every reply starts with <think>, so the first generated tokens look the same whether the model will refuse or not. First-token scoring and prefix checks therefore tell you very little. This release measures the refusal contrast inside the reasoning and judges results on the final answer.

1. Prompts. Pinned splits from Arditi et al.'s refusal_direction at revision 9d852fa, with hashes verified.
2. Baseline reasoning. The original Q4_K_M generated 192 greedy tokens of reasoning for 160 harmful and 160 harmless training prompts. Refusal language appeared in 145 harmful and 3 harmless reasonings. Of the generated output, only token IDs and that flag were kept.
3. Contrast. A small layer-streamed forward pass ran those tokens through the original BF16 weights, without the repository's remote code. At each layer, the residual was averaged over reasoning tokens: 145 harmful prompts voicing refusal minus 145 harmless prompts that did not.
4. Directions. For layers 16 to 78 (zero-based), the unit contrast was averaged with its neighbours within ±2 layers, then renormalized. The token embeddings use layer 16's direction.
5. Projection at strength 1.0. Ripple's layers add a normalized residual, w * out / rms(out), where w is the post-norm weight. The axis to remove from the output matrices is therefore U, the normalized d * w, giving W' = W - U U^T W on attn_output and ffn_down. The embedding table gets E' = E - (E d) d^T. The largest remaining component was 2.69e-8.
6. Patch in place. The edit was written straight into the BF16 GGUF, changing 127 tensors: the embedding plus 2 for each of 63 layers. The patcher checks existing bytes before writing, and a baseline round trip reproduced the file byte for byte. Layers 0 to 15 and 79 are untouched.
7. Quantize. Q4_K_M, Q5_K_M and Q6_K use an importance matrix computed on the edited Q8_0, from 829 harmless chat prompts and 11 llama.cpp source files. The corpus contains no harmful text. Q8_0 has no importance matrix.

The direction vectors are in abliteration_directions.safetensors, and the exact specification is in abliteration.json, so you can inspect the edit or reapply it to the source weights.

Known limits

• This edit is the first candidate that passed, not the winner of a search. It cleared validation and the functional suites on the first try, so the other queued variants were not run. No other layer range or strength was evaluated, and a better setting may exist.
• The capture pass was not numerically compared with llama.cpp. The streaming forward pass produced the directions, while every result above comes from llama.cpp. Those results are the evidence that the edit works, not any match between the two.
• Only this one edit is measured, on one machine. CPU, Metal, single-GPU and other backends are untested, as are agentic coding sessions like those the upstream card benchmarks.
• Upstream's NVFP4 and DFlash2 releases are not part of this build. Speculative decoding with DFlash2 was not tried with these files.
• This repository has no native safetensors checkpoint. It contains GGUF files only.

Files

Attribution

Original weights and implementation: Ricursive Intelligence, Apache-2.0, source revision 4f872a1ef1376a1962c56bab6d874d017be7eb96. Refusal-direction method and prompt splits: Arditi et al., Refusal in Language Models Is Mediated by a Single Direction. Runtime and quantization: llama.cpp, MIT. Perplexity text: WikiText-2 test set, through ggml-org/ci. Build code was generated with AI assistance and tested as recorded here. Please cite the upstream model:

@misc{ricursive2026ripple65b,
      title={Ripple-65B: Pushing the Frontier of Local Intelligence},
      author={Ricursive Intelligence},
      year={2026},
}