license: mit
language:

• en
• zh
  pipeline_tag: text-generation
  library_name: gguf
  base_model: orcarouter/DeepSeek-V4.1-Flash-1017-abliterated
  base_model_relation: quantized
  tags:
• gguf
• llama.cpp
• local
• deepseek
• deepseek-v4
• deepseek-v4.1-flash
• deepseek-v4.1-flash-1017
• moe
• engram
• mixed-precision
• imatrix
• abliterated
• uncensored
• ai-red-team
• reasoning
• function-calling

DeepSeek-V4.1-Flash-1017-abliterated GGUF

The 763B uncensored DeepSeek-V4.1-Flash-1017, on one box.

236 GB in memory · 197B-parameter Engram streamed from your SSD · llama.cpp

OrcaRouter AI Gateway · X · Discord · GitHub · All Models

The full DeepSeek-V4.1-Flash-1017, run locally.

These are GGUF builds of the abliterated (refusal-removed)
DeepSeek-V4.1-Flash-1017.
The model is a 763B-parameter MoE with 384 routed experts (top-6), a 197B-parameter Engram n-gram
memory, compressed sparse attention and a 1M-token context. The routed experts are the only part
the bits were spent on: every layer's experts got their own precision from measured KL
divergence, not a hand-set rule.

The Engram table is a lookup that reads 48 rows per token. llama.cpp maps it from disk and reads
it on demand, so it lives on your SSD, not in RAM. What has to fit in memory is everything else.

⚠️ Disclaimer — read before use

This model is built on an abliterated base whose safety alignment has been substantially reduced:
the refusal direction is orthogonalized out of every matrix that writes the residual stream. It will
comply with many harmful, unethical, offensive, or illegal requests that the original
DeepSeek-V4.1-Flash-1017 would refuse.

• It is released strictly for legitimate research: interpretability, AI-safety and
  refusal-mechanism study, red-teaming, and robustness evaluation.
• You assume full responsibility for how you use it and for everything it generates. Add your
  own safety, moderation, and abuse-prevention layers before any deployment.
• Use must comply with the MIT License inherited from the base
  model and with all applicable law.
• The authors accept no liability for misuse. Its outputs do not reflect the views of the
  uploaders or of DeepSeek.

Pick a build

Mix-256G is the build to use. It needs a box with 256 GB of RAM, unified memory, or VRAM. At 3.36
bits per expert weight, the model's next-token choice matches the unquantized model on 93.7% of
held-out tokens.

Mix-128G fits 128 GB, but it is not a substitute. All experts at llama.cpp's smallest type
(IQ1_S) already take 111 GB, so a 128 GB box is at the floor: perplexity is 55% higher and one token
in five differs from the original. Use it to try the model, not to rely on it.

In memory is everything except the Engram table. Download adds the 110.6 GB Engram table, which
stays on disk.

Evaluation

Both builds were scored against the lossless GGUF of the same checkpoint (MXFP4 experts, BF16
dense, MXFP8 Engram, converted bit-exact) on held-out text: wikitext-2 test, characters [0, 300000),
32 windows of 2048 tokens, full-vocabulary KLD. The precision search never saw this text. Each
build is compared with llama.cpp's own mixture at a similar size, quantized from the same
weights with the same importance matrix:

Measured with the Engram table at MXFP8. The released files carry it at IQ4_NL (110.6 GB
instead of 202.8 GB). The swap was not rescored. Per
unsloth's measurements on this model,
the Engram precision barely moves top-1.

Method note. These are full-vocabulary KLD and next-token agreement on held-out text at 2k
context, not a downstream benchmark.

Refusal and capability of the abliterated model

Measured on the unquantized abliterated checkpoint
(orcarouter/DeepSeek-V4.1-Flash-1017-abliterated),
not re-measured on these GGUF builds. They are copied here from that model's card. The KLD above
is how far each build sits from that checkpoint.

Setup.

• Hardware and runtime: 8×H200, the checkpoint's own reference implementation under tensor
  parallelism.
• Decoding: greedy, thinking_mode="chat", 64 new tokens.
• Scoring: refusal is judged by a rule-based opening-phrase classifier. caveat means the model
  answered but wrapped the answer in a disclaimer. Treat the numbers as indicative, not
  publication-grade.

Safety: harmful-prompt refusal (lower = more uncensored). Weighted mean over all 500 prompts:
0.074, against 0.874 for the base checkpoint.

Over-refusal: benign prompts wrongly refused (lower = better). Weighted mean over all 350
prompts: 0.006, against 0.069 for the base checkpoint.

Capability retention.

MMLU is unchanged; the one-point difference is within the sampling error of 300 questions. It is the
only capability measured.

How the precision was chosen

Each tier was searched separately, with the budget set on the bytes that have to be in memory:

1. Measure. For every group of tensors, the search measures the real KLD of moving it one
   precision level up and one level down. A group is one layer's experts, with gate+up and down
   kept apart, or one kind of dense matrix. Each measurement runs llama.cpp's own kernels.
2. Choose. A knapsack picks the combination of moves that fits the budget exactly.
3. Accept. Only steps whose candidate KLD was actually measured lower are kept. The search
   repeats until it converges.

The searched 256G build puts its bits where they count:

In the same budget, a uniform allocation scores KLD 0.146 and the searched one 0.045 (in-sample).
Starting from the uniform allocation, upgrading any single one of the middle layers to MXFP4
removes ~0.04 KLD. Their errors compound downstream rather than add up.

In the 128G build there is almost nothing left to place. Experts are IQ1_S everywhere except a few
later layers at IQ1_M or IQ2_XXS.

The other parts are set as follows:

• Attention and shared experts are Q5_K–Q8_0 in 256G and Q4_K–Q6_K in 128G.
• The router, norms and hyper-connection mixers are left as shipped.
• The importance matrix comes from 614k tokens of English/Chinese chat and Wikipedia.

What's in this repo

Both builds are text-only: the vision tower (mmproj) and the DSpark speculative drafter are not
included. The DeepSeek-V4.1 chat template is embedded in each build.

Serving (llama.cpp)

DeepSeek-V4.1 is not in llama.cpp master yet. Build the PR that adds it,
ggml-org/llama.cpp#30817. These files were made
and tested at commit b72050d03:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch origin pull/30817/head:pr-30817 && git checkout b72050d03
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server llama-cli

hf download orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF --include "Mix-256G/*" --local-dir ./dsv41-gguf

# GPUs with ~250 GB of VRAM in total (tested on 2 x H200)
llama-server -m dsv41-gguf/Mix-256G/DeepSeek-V4.1-Flash-1017-abliterated-Mix-256G-00001-of-00007.gguf \
  -ngl 99 -c 32768 -np 1 --no-reasoning-preserve

• Keep the files on an NVMe SSD and leave mmap on (the default). The Engram table is always
  kept off the GPU and read lazily. --mlock or --no-mmap would pull all 110 GB into RAM.
• 256 GB of system RAM and one GPU: add --cpu-moe to keep the routed experts in RAM and run
  attention on the GPU. This setup was not measured here.
• --no-reasoning-preserve drops earlier turns' thinking from the prompt, which is what
  DeepSeek's reference encoder does. With it, llama.cpp's rendered prompts match encoding.py
  token for token on DeepSeek's test cases.
• Thinking is on by default. Pass "chat_template_kwargs": {"thinking_mode": "chat"} for a direct
  answer, or "reasoning_effort": "low" | "medium" | "high" | "max" (or 1–100) to set its depth.
  The reasoning comes back in message.reasoning_content.
• Set a system prompt. Without one, the model sometimes misnames itself in thinking mode.

Measured on H200 (CUDA), short prompts, one request at a time:

CPU-only, Apple Metal and partial-offload speeds were not measured.

Model details

Bias, risks, and limitations

• Safety guardrails are substantially reduced (inherited from the abliterated base); see the
  disclaimer. The refusal and MMLU numbers above come from the unquantized checkpoint. Mix-128G in
  particular sits far enough from it (KLD 0.52) that its behaviour may differ.
• Mix-128G is heavily degraded. This is the limit of 1.6-bit post-training quantization on this
  model, not a tuning issue.
• The IQ4_NL Engram in the released files was not rescored. The table above was measured with
  MXFP8.
• Text-only. There is no vision projector and no DSpark drafter.
• Inherits the biases and limitations of DeepSeek-V4.1-Flash-1017. Long-context behaviour was not
  tested: all measurements are at 2k context or on short chats.
• Requires a llama.cpp build with PR #30817. Stock llama.cpp, Ollama and LM Studio will not load
  these files until it is merged.

License

MIT, inherited from the base model
deepseek-ai/DeepSeek-V4.1-Flash-1017.
The embedded chat template is unsloth's port of DeepSeek's encoder
(unsloth/DeepSeek-V4.1-Flash-1017-GGUF, MIT).

One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website · Models · GitHub · Discord · X