license: other
license_name: deepseek
license_link: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731/blob/main/LICENSE
base_model: deepseek-ai/DeepSeek-V4-Flash-0731
tags:

• uncensored
• abliterated
• abliteration
• refusal-direction
• activation-steering
• vllm
• deepseek
• interpretability
  library_name: safetensors

DeepSeek-V4-Flash-0731 — Uncensored / Abliterated, switchable at runtime

An uncensored (abliterated) DeepSeek-V4-Flash without a second checkpoint, without
reloading the model, and without swapping anything on disk.

Abliteration normally means downloading a whole separate ~157 GB model with the refusal
behaviour permanently burned into its weights. This is the same effect delivered a different
way: a 757 KB file of direction vectors sits beside your existing checkpoint, and the
uncensoring strength becomes a dial you turn at runtime — one HTTP call, effective on the
next request, no restart, no reload, no downtime.

curl -XPOST localhost:8888/admin/refusal_lambda -d '{"lambda": 1.5}'   # uncensored
curl -XPOST localhost:8888/admin/refusal_lambda -d '{"lambda": 0}'     # back to stock

At λ=1.5 the model refuses 0 out of 10 prompts it declined 9 out of 10 times at stock —
with no measurable loss in speculative-decoding acceptance, long-context retrieval, or
tool-calling. At λ=0 it is bit-exact to the unmodified DeepSeek release, so "off" is
genuinely off rather than approximately off.

Because it is a dial rather than a swap, it reaches an operating point no published
abliterated checkpoint can offer: the baked edit does not merely remove the refusal
direction, it overshoots and inverts it (~240 %), and that overshoot is what costs quality.
The clean range in between only exists at runtime.

Code, patches, benchmark harnesses and every raw result:
https://github.com/pocharlies/deepseek-v4-flash-rank1-refusal-projection

vLLM 0.27.1 DSpark runtime — 2026-08-22

The exact GB10 compatibility sources are published in GitHub under
runtime/vllm-0.27.1/.
The serving configuration was not retuned: DSpark probabilistic k=5, TP=2, B12X,
nvfp4_ds_mla, 393216 context, 4 sequences and 8192 batched tokens. The port recompiles
the known-good DeepGEMM sm_121 source against vLLM 0.27.1's PyTorch ABI and carries the
multimodal prompt-update compatibility override.

The measured code workload reached 67.41 tok/s median, 0.35 s TTFT and 67.39% DSpark
acceptance. Health, min_p, tool calling and image input passed with zero restarts.
Machine-readable evidence is in
benchmarks/2026-08-22/vllm-0.27.1-dspark.json.

Measured on 2× DGX Spark GB10 (sm_121), vLLM 0.25.2, TP=2, DSpark speculative decoding k=5,
--max-model-len 262144, --enable-prefix-caching.

The result

83.4 minutes of alternated A/B on the same pod, same day, same load. Every number above is
backed by raw JSON in the GitHub repo under bench/results/.

The baked checkpoint cannot reach this operating point at all. Measured on the weights,
it sits at λ_eff ≈ 2.43 — it does not remove the direction, it inverts it (see below). Prior
measurement of that checkpoint on this deployment put acceptance at 0.5128, below the 0.55
floor required here.

In plain terms: the advantages over a normal abliterated model

The usual way to run an abliterated model: you download a second, complete copy of the
model — a separate ~157 GB checkpoint. Now you keep two copies on disk (~313 GB), and they are
mutually exclusive on the same GPUs. Want the normal model back? Stop the server, load 157 GB
of different weights across both nodes, wait for it to come up. Every in-flight request dies.
Want to compare the two on the same prompt? You do it hours apart, on different server
processes, and hope nothing else drifted in between.

This way: you download nothing extra. Your existing checkpoint stays exactly as it is, and
this 757 KB file sits next to it — about 2,000× smaller than the published 1.54 GB overlay,
and roughly 220,000× smaller than a second copy of the model. One server process. Switching is
one HTTP call:

curl -XPOST localhost:8888/admin/refusal_lambda -d '{"lambda": 1.5}'   # ablation on
curl -XPOST localhost:8888/admin/refusal_lambda -d '{"lambda": 0}'     # ablation off

It takes effect on the next request. No restart. No reload. No second copy on disk. No
downtime, and nothing to re-download.

Three things that follow

1 · "Off" genuinely means off. λ=0 is bit-exact to the unmodified model — verified with
torch.equal, not "close enough". You are not permanently running a modified model and hoping
it behaves like the original on ordinary work; at zero it is the original. That is why the
base weights stay byte-identical to the DeepSeek release and can be checked with sha256.

2 · It is a dial, not a switch. λ=0 is stock, λ=1.5 removes refusal entirely, everything in
between is available, and λ can go negative — making the model more reticent than stock,
which no abliterated checkpoint can offer.

3 · A/B testing becomes honest. Both arms run on the same process, same weights, same hour,
same load; you just flip the dial between runs. That is how the numbers above were produced —
6 alternated runs per arm in one 83.4-minute session. With two separate checkpoints you cannot
do this, and the drift is real: two λ=0 runs in that same session came in at 57.87 and
40.52 tok/s.

The two honest costs

Switching is instant, but the prefix cache for the new λ starts cold. λ is part of the
block hash key deliberately — reusing cached blocks across different λ would silently corrupt
state, and that is the single most likely way to get this wrong. So the first request after a
switch re-prefills its context. Measured on this hardware: 76.3 s for a 136,879-token
prompt, ~12.6 s at 16.7k tokens, negligible for short prompts. Steady-state traffic at a
fixed λ caches normally and is unaffected.

λ=0 is free in output, not in compute. The dot product and subtraction run in all 46 layers
on every token regardless of λ, because branching on λ is exactly what breaks CUDA graph
capture. For genuinely zero overhead, unset VLLM_REFUSAL_DIRS and restart.

What is in this repository

refusal_dirs.safetensors    757,712 bytes — 46 tensors, each [4096] float32, unit norm

Metadata embedded in the file:

Keys are module names: layers.0.attn.wo_b … layers.42.attn.wo_b, mtp.0-2.attn.wo_b.

These are not weights. They are the normalized principal direction of ΔW between two
publicly released checkpoints. They do nothing on their own — they need the runtime hook
from the GitHub repo.

How it works

Editing a weight and projecting the sublayer output are the same function:

(W − λ·r̂r̂ᵀW)·x  ≡  W·x − λ·r̂·(r̂ᵀ·W·x)

The left side is what abliterated checkpoints ship. The right side needs only r̂ — one unit
vector in ℝ⁴⁰⁹⁶ per edited module — and never touches W.

Verified to machine precision in float64 across 4 modules: max relative error 8.8e-16 to
9.3e-16. At λ=0 the hook is bit-exact to the base model (torch.equal).

So: 757 KB instead of 1.54 GB, base weights verifiable by sha256 against the DeepSeek
release, and λ as a continuous runtime parameter rather than a property frozen into a
checkpoint.

Method: Arditi et al., Refusal in Language Models Is Mediated by a Single Direction
(NeurIPS 2024). Directions extracted from cebeuq/DeepSeek-V4-Flash-0731-abliterated.

The finding that reframed the project

The plan assumed the baked edit hits a ~68 % ceiling from the FP8 round-trip and
therefore over-projects to λ=2.5 to compensate. Measured directly on the weights:

There is no ceiling. The published checkpoint does not fall short — it removes ~240 % of the
direction, i.e. inverts it and leaves it at ~1.4× its original magnitude pointing the
opposite way.

λ=2.5 compensates for nothing; it is the overshoot. The degradation measured on the baked
deployment (acceptance 0.5128, code throughput 46.6–55.3 vs 59.32 tok/s) is the cost of
inverting the direction, not of removing it.

The whole useful range 0 < λ ≤ 1.5 is territory no published checkpoint occupies.
Baked abliteration jumps from +100 % (unedited) straight to −143 %. The clean point between
them only exists as a runtime dial.

At λ=1 the residual component is exactly 0 in exact arithmetic; with bf16 activations the
real residual is ~1.7e-3 relative. That is 99.83 % removal, not 100 %.

Extraction gates — two passed, two failed

Gate 2 failed and r=2 does not fix it. s₁/s₂ is 13–23×, but from s₂ to s₈ the decay is
only 2.5–2.7× — a flat tail. r=2 buys 0.36 points of energy; r=8 still doesn't reach 0.91.
There is no second direction: the missing ~10 % is broadband requantization noise from E4M3
with power-of-two block scales — which is precisely the artifact this design avoids, not
something the hook should reproduce.

Also measured: the edit subtracts in all 46 modules, and cos(v₀, u₀ᵀW_base) ≥ 0.9873
across all 46 — the dominant component genuinely has the shape of a projection.

Backbone and MTP are edited differently (δ 0.0615 vs 0.0410, λ_eff 2.435 vs 2.343). In the
runtime hook they share one dial and stay aligned by construction; in the published
checkpoint they do not, and that misalignment is a prime suspect for part of the acceptance
drop.

Integration — the five things that silently break it

Full patch in the GitHub repo (patches/0001-rank1-projection.patch, 460 lines, 7 files,
against vLLM 0.25.2.dev0+g752a3a504).

1. λ must be a device tensor, never a Python float. With CUDA graph capture a Python
   scalar gets baked into the captured graph — changing it does nothing, with no error. One
   tensor per device, shared across layers, mutated with fill_().
2. λ must enter the prefix-cache hash key. A prefix cached at λ=0 and reused at λ=1.5
   gives corrupt state, silently. The right precedent is cache_salt, not lora_int_id. λ
   quantized to integer (×1000).
3. The setter must go through collective_rpc. With the mp executor a local setter
   reaches no worker rank. Divergent ranks must be a 500, not a 200.
4. Apply at the sublayer output, before hc_post. The mHC write is rank-1 across
   streams, so removing the direction at the sublayer output clears all four streams at once.
   Do not project onto the [B,S,4,4096] carrier.
5. TP needs nothing special. wo_b is RowParallelLinear with reduce_results default;
   its return is already post-all-reduce, and the projection is linear.

Kernel shape — do the dot product in fp32 even when y is bf16 (1.66e-3 vs 2.29e-3 median
error; it's free, though both are dominated by bf16 storage of y):

proj = (y.float() @ r_hat)                       # [num_tokens]
y = y - (lam * proj).unsqueeze(-1) * r_hat       # lam is a device tensor

Control surface: POST /admin/refusal_lambda {"lambda": 1.5}. The router only mounts when
VLLM_REFUSAL_DIRS is set. Keep it off the public ingress — vLLM cannot enforce that.

Production serving note — 2026-08-20: --long-prefill-token-threshold 1024

The 2× DGX Spark TP=2 deployment this card is measured on now runs with
--long-prefill-token-threshold 1024 (vLLM V1 SchedulerConfig). On this build the flag
caps the per-step chunk of any prefill larger than the threshold, so a single 80K-token
prompt can no longer monopolize the 8,192-token step budget while other requests decode
(verified via vllm:iteration_tokens_total: 99.2 % of steps ≤1,024 tokens after the change).

Measured on the production pair, same traffic source, 2 h after vs 16 h before: median
generation throughput at 2–6 concurrent requests went from 2.0–5.3 tok/s to 12.2–53.8 tok/s
(2.9–10×), max queue depth 37 → 7, TTFT under mixed load 43–114 s → 0.18–0.33 s. DSpark
acceptance is unchanged (54.6 %, mean accepted length 3.71). The cost: an 80K prefill now
takes ~59 s (~1,363 tok/s) instead of ~20 s with 8K chunks. The λ projection is orthogonal
to the scheduler — nothing about the dial changes.

Calibration

Every figure in this table has per-run raw JSON in the GitHub repo under bench/results/.
The λ=0 and λ=1.5 acceptance arms were run alternated in a single 83.4-minute session
(compare_full.log), which is what makes them comparable — this box drifts, and two λ=0 runs
in the same session came in at 57.87 and 40.52 tok/s.

On the λ=1 arm and the baked baseline. An earlier working report
(docs/projection-validation.md) records acceptance 0.5635 at λ=1 and 0.5128 for the baked
checkpoint at λ_eff ≈ 2.43. Both are real prior measurements, but their per-run JSON is not in
the published set, and that report's λ=1.5 figure (n=3) was superseded five hours later by the
n=6 alternated run above. Treat the λ=1 and baked numbers as indicative, not as evidence of
the same weight as the rest of this table. The λ=1 refusal rate is separately backed by
refusal_rate.json and refusal_sweep.json.

It saturates at 1.5. λ=2 adds nothing over 1.5: refusal is already 0 % there. 1.5 is the
operating point.

2026-08-19 — λ up to 2.5, measured on capability and on truthfulness

Recommending 1.5 used to rest partly on the assumption that higher λ degrades. That
assumption was tested and it does not hold on this checkpoint:

Paired, temperature 0, 4000-token budget, per-request λ (the global dial stayed at 0 and was
re-checked between arms). No comparison is significant — every McNemar p ≥ 0.09, and the
curve is not monotonic. 0 yields out of 113 sycophancy opportunities; 90/90 needles.

So the honest statement is: stay at 1.5 because above it you gain nothing, not because it
breaks. Two caveats that keep this from being a licence to raise it:

• The truthfulness suites return 100 % in every arm — a ceiling effect. They refute the
  hypothesis that ablation also removes "no, that is false", but a saturated suite cannot
  rank arms. With 0 events in ~22 items per arm, the rule of three only rules out sycophancy
  above ~13.6 %.
• Acceptance at λ≥2 is still not measured. The baked-checkpoint figure at λ_eff ≈ 2.43
  (0.5128) remains the only signal there and it is below the floor. Capability and speculative
  acceptance are different axes; nothing above speaks to the second.

Full report, raw JSON, and the confounded first version of the quality sweep (kept
deliberately as a worked example): benchmarks/2026-08-19/.

What this does not establish

• General capability is unmeasured. Closed 2026-08-19 for MMLU-Pro and GSM8K
  across λ ∈ {0, 1, 1.5, 2, 2.5} — see Calibration. HumanEval is still not run.
• Nothing here validates an answer that only a high λ produces. That content is, by
  construction, what the model represented least well, and it is the one domain where no
  ground truth exists to check it against. This model can correct textbook myths and retrieve
  needles at 128k and still confabulate a fluent, confident, structurally plausible
  procedure. Fluency is not a correctness signal — do not read the tables above as a warrant
  for trusting output that only appears once λ is raised.
• 256k is unmeasured, deliberately. Retrieval is validated to 126,940 real tokens.
• Variance rises with λ even where the mean passes. 1 of 6 runs at λ=1.5 and 2 of 6 at
  λ=1 dipped below the 0.55 acceptance floor. Means are indistinguishable; individual runs
  are not always.
• λ=0 is bit-exact in output but not free in compute. The dot product runs in all 46
  layers per token regardless of λ. For zero cost, unset VLLM_REFUSAL_DIRS and restart.

Benchmarks that lied before being fixed, documented in full in the repo: the model spends
1,100–1,400 tokens reasoning before answering, so a low max_tokens returns empty
content and reads as a spectacular refusal-removal result — it isn't. And exact string match
is not a valid gate: vLLM is not deterministic run-to-run at temperature 0, with measured
noise of ±1 cell in 30.

That first failure recurred on 2026-08-19, in a different suite and wearing a p-value: the
quality sweep at max_gen_toks=2048 produced a tidy descending curve and two "significant"
results, and all of it was truncation — empty answers rise with λ (18→31) because the model
reasons longer, and empties score as wrong. Conditioned on non-empty answers the accuracy is
flat at every λ. On a reasoning model, a short generation budget confounds λ with
verbosity. The confounded run is published next to the corrected one.

Intended use and a real caveat

This is interpretability and serving-infrastructure work: it makes an already-published
weight edit auditable, reversible and continuously adjustable, and it measures what that edit
actually costs — which the published checkpoint did not.

The uncomfortable part, stated because it is part of the engineering picture:

Reducing a model's resistance to instructions reduces its resistance to injected
instructions. Prompt injection and refusal route through overlapping machinery. A clean
λ=1.5 yields a model more capable and more completely stripped of its ability to decline
than the baked checkpoint, whose clumsiness incidentally limited how useful it was. The
better the dial works, the more the isolation matters.

If you wire this to tools with write access or feed it untrusted scraped content:

• λ>0 should not share credentials with write-capable tools — use a separate deployment.
• Restrict the toolset when λ>0 to read-only paths.
• Keep /admin/refusal_lambda off the public ingress.

One measured nuance worth keeping: at λ=1.5 the model still refuses to claim it issued a
refund it cannot issue (tool-calling 8/8 includes that test). The projection removes the
policy refusal, not the grounded one.

Citation

@misc{arditi2024refusal,
  title  = {Refusal in Language Models Is Mediated by a Single Direction},
  author = {Arditi, Andy and Obeso, Oscar and Syed, Aaquib and Paleka, Daniel and
            Panickssery, Nina and Gurnee, Wes and Nanda, Neel},
  year   = {2024},
  note   = {NeurIPS 2024},
  eprint = {2406.11717},
  archivePrefix = {arXiv}
}

Directions derived from the difference between deepseek-ai/DeepSeek-V4-Flash-0731 and
cebeuq/DeepSeek-V4-Flash-0731-abliterated; they inherit the DeepSeek Model License. Code in
the linked GitHub repository is Apache-2.0.