license: apache-2.0
base_model:

• openbmb/MiniCPM-V-4.7-35B-A3B
  base_model_relation: finetune
  quantized_by: AtomicChat
  pipeline_tag: image-text-to-text
  library_name: gguf
  tags:
• atomic-chat
• minicpm
• minicpm-v
• openbmb
• moe
• gguf
• imatrix
• llama.cpp
• lora
• abliteration
• refusal-direction
• not-for-all-audiences

How to Run MiniCPM-V 4.7 35B-A3B Without Refusals Locally

One refusal direction projected out of OpenBMB's original weights, shipped two ways: three baked GGUF builds, and a 106 MB LoRA adapter for the MiniCPM-V 4.7 quants you already have. The measurements behind this card are public.

• On held-out prompts, refusals fall from 96.3% to 0% in English and from 70% to 0% in Russian, and from 13 of 30 to none with thinking on. MMLU moves by 0.25 points, inside the noise.
• The 20.1 GB build matches the original's token choice 94.5% of the time; our quant of the unmodified model at the same size does 94.9%.
• Already running our MiniCPM-V 4.7 quants? Add the adapter with --lora-scaled instead of downloading a new model.

The builds

AD-Q4_K-IQ4_XS is the one for a 24 GB card, with room for a long context; take
AD-Q6_K if it fits.

KLD and top-1 are against the original, unmodified BF16 model, so they hold
the ablation and the quantization together. For scale: the ablation alone, on
BF16 weights, is 0.0090; our plain Q8_0 of the original is 0.0051. The same
layouts of the unmodified model, from our quants:

The names follow our original quants:
AD-<ffn_down>-<ffn_up> is what the routed experts got; attention, the shared
expert, the embedding and the output head stay at q8_0.

What changed, and what did not

Original against ablated, both BF16, on prompts never used to pick anything:

Refusal is counted by the opening of the reply, so it is indicative, not a
judge; empty or degenerate replies count as damage, and there were none. The
baked builds were checked the same way on the same test prompts.

The adapter

lora/MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-LoRA.gguf: rank 1, 106 MB, 120
sites. It applies to any GGUF of the original MiniCPM-V-4.7-35B-A3B weights,
including our published quants. Use scale 1.0:

llama-server -m MiniCPM-V-4.7-35B-A3B-AD-Q4_K-IQ4_XS.gguf -ngl 99 --jinja \
  --lora-scaled lora/MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-LoRA.gguf:1.0

The adapter was measured on the BF16 weights (the table above); on a quant it
adds the same edit on top of that quant's own error.

Get started

• llama.cpp at or after 69f201a (MiniCPM-V 4.7 support, ggml-org/llama.cpp#29416).
  Download one build and point -m at its first shard:

hf download AtomicChat/MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-GGUF \
  --include "MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-AD-Q4_K-IQ4_XS/*" --local-dir .

llama-server \
  -m MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-AD-Q4_K-IQ4_XS/MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-AD-Q4_K-IQ4_XS-00001-of-00011.gguf \
  --mmproj mmproj-MiniCPM-V-4.7-35B-A3B-Abliterated-Uncensored-F16.gguf \
  -ngl 99 -c 32768 --jinja

With --jinja the model's own chat template starts every answer in thinking
mode; turn it off per request with "chat_template_kwargs": {"enable_thinking": false}.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt
   token, chat template applied, thinking off: 416 harmful against 416 harmless
   English prompts, taken entering block 28 of 40, with the harmless-mean
   component removed.
2. Edit. W' = W - r rᵀW on the matrices that write into the residual: 40
   attention and linear-attention outputs, the down projections of all 256
   experts in all 40 MoE blocks, and 40 shared experts. The token embedding is
   left alone: the vision projector copies rows of it.
3. Choice. Directions from four depths at four strengths, then narrower
   writer sets, one direction per block, English only or English + Russian,
   and a random direction as the control, all on validation prompts; the
   numbers on this card come from held-out test prompts. The screen's own
   pick (one direction per block, scale 0.75) drifted a little less on the
   first token but still refused 5 of 30 prompts with thinking on, which this
   template does by default, so this release uses the English direction at
   scale 1.0, which refused none.
4. Bake. The same edit applied to the BF16 weights in f32 and rounded once;
   every one of the 120 writers checked to keep none of its component along
   r, within 7.6e-3 after bf16 rounding. Then quantized with the importance
   matrix of our original quants,
   which a 300-chunk matrix of the ablated model agrees with (cosine 0.999 or more).

Everything was measured against one reference, one corpus, one machine: the
original BF16 model's own logits over a held-out neutral set, 87 chunks at 4096
context, llama.cpp 8bb3cf0d, 2x RTX PRO 6000 (sm_120). The unmodified quants
above were measured on H100s against their own reference from the same weights;
our Q8_0 scores 0.0051 on both.

Limitations

• The refusal count reads the opening of each reply; a soft refusal phrased as
  an answer would be missed.
• Vision was not tested after the edit. The mmproj files here are the
  original's projector, unmodified.
• Not run yet on a Mac, in LM Studio or in Ollama.
• The direction comes from English prompts at one token position. Russian
  refusals went with it; other languages were not measured.

Responsible use

This model has a learned refusal direction removed, so it will answer requests
the original declines. Nothing about it makes the output safe, correct or
lawful, and any deployment needs its own access controls and policy
enforcement.

Credits

Direction estimation, adapter build, quantization and evaluation by
AtomicChat. Base model MiniCPM-V 4.7 35B-A3B by
OpenBMB, under Apache-2.0. The method follows
Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).