license: mit
base_model:
- deepseek-ai/DeepSeek-V4.1-Flash-1017
pipeline_tag: text-generation
tags: - abliteration
- uncensored
- moe
- thinking
- engram
arxiv: 2406.11717
DeepSeek-V4.1-Flash-1017 Abliterated (transferred directions, scale 3.0)
Responsible use
This is an uncensored research model. Its built-in refusals were removed, so it answers prompts the base model declines, including chemical and biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming, offensive-security research, and refusal-rate evaluation. It has no guardrails of its own: if you deploy it, add your own input and output moderation (for example Llama Guard).
What this is
Abliterated variant of deepseek-ai/DeepSeek-V4.1-Flash-1017 (NBBB backbone / 8-16B active MoE with the 2-layer Engram n-gram memory). The refusal directions were not re-measured on this checkpoint: they were transferred from our DeepSeek-V4.1-Flash-abliterated measurements (both models share the 40-layer, dim-5120 residual trunk), then applied as norm-preserving biprojected edits at scale 3.0 to the attention output projection (attn.wo_b) and the shared-expert down projection (ffn.shared_experts.w2) across layers 4-36, exactly as in the original run.
The repo mirrors the upstream checkpoint (same FP8/FP4 layout, same tokenizer). Only the abliterated tensors differ; everything else is byte-identical to upstream. refusal_directions.pt holds the transferred per-layer directions, so you can re-ablate at any scale in seconds without re-measuring. abliteration_config.json records the final parameters and evaluation results.
Load
The deepseek_v41 architecture is not in transformers/vLLM mainline (as of Oct 2026); use DeepSeek's reference runtime with its convert.py, or SGLang dev-dsv41:
# reference runtime: convert to the TP-sharded format (fp4 experts kept as shipped)
python convert.py --hf-ckpt-path ./DeepSeek-V4.1-Flash-1017-abliterated \
--save-path abl-tp8 --model-parallel 8 --expert-dtype fp4
torchrun --nproc-per-node 8 inference/generate.py \
--ckpt-path abl-tp8 --config config.json --interactive
On 80GB GPUs the Engram tables must live in host RAM; the reference runtime in this repo's pipeline/ was patched (ABL_ENGRAM_HOST=1) to keep the ~25GB/rank fp8 tables on CPU and dequantize on the GPU.
Evaluation (scale 3.0)
The standard abliteration protocol (the one Heretic made automatic): held-out harmful prompts with normalized keyword detection plus an LLM judge, KL divergence vs base on harmless prompts, and a GSM8K spot check. Eval mode: chat, greedy, TP8 on H100 (fp4 experts, host-RAM Engram).
| Metric | Base | Abliterated 3.0 |
|---|---|---|
| Refusals (normalized keyword detector, X/100) | 99 | 46 |
| True refusals (LLM judge) | not judged | 0 |
| Judge: COMPLIANT / PARTIAL | not judged | 53 / 47 |
| KL divergence (100 harmless) | 0 | 0.093 |
| GSM8K (20-problem spot check) | 19/20 | 18/20 |
The honest number is the judge's 0/100 true refusals: every prompt was answered, 53 cleanly and 47 with a hedge or disclaimer before complying. The keyword detector's 46 over-counts those hedges exactly the way it did on the original V4.1-Flash release (keyword 41 / judge 1 there); read the raw responses in eval-results/ before quoting either number. GSM8K 18/20 vs the base's 19/20 is within run-to-run noise; KL 0.093 on harmless prompts sits low but above the identity noise floor measured on the original release (0.131 there, different judge setup).
Method
Per-layer refusal directions were measured at the hyper-connection-collapsed residual stream on 128 harmful vs 128 harmless prompts (on the V4.1-Flash base of this lineage), orthogonalized against the harmless mean direction (projected abliteration), then applied as norm-preserving biprojected edits to wo_b and shared_experts.w2 rows across layers 4-36. FP8 32×32 (ue8m0) blocks are dequantized → ablated → requantized exactly (power-of-two scales), leaving the FP4 routed experts, Engram memory, and vision tower byte-identical to upstream.
MIT, inherited from the base model. Base model: deepseek-ai/DeepSeek-V4.1-Flash-1017. Abliteration follows the residual-direction method (Arditi et al., Refusal in LLMs is mediated by a single direction, NeurIPS 2024). Not affiliated with or endorsed by the base-model authors.