library_name: vllm
license: mit
base_model:

• deepseek-ai/DeepSeek-V4.1-Flash-1017
• pqhaz/DeepSeek-V4.1-Flash-1017-heretic
  tags:
• deepseek
• mxfp4
• text-generation
• tool-use
  pipeline_tag: text-generation

Visual comparisons: Native mixed MXFP4

Native mixed MXFP4 · NVFP4 target · EXL3 2.875-bit experts · EXL3 3.5-bit experts

Four formats, one checkpoint family. Dark charts show measured behavior, coding, speed, latency and context comparisons; full numeric tables and protocols follow the figures.

Did the edit change harmful-response behavior?

300 HarmBench prompts. Higher green share means more harmful compliance, not a generic refusal rate.

JailbreakBench prompts: chat-mode comparison

100 JBB prompts; auxiliary HarmBench grading, not official JBB scoring. Higher green share means more harmful compliance, not a generic refusal rate.

JailbreakBench prompts: reasoning-mode comparison

100 JBB prompts; auxiliary HarmBench grading, not official JBB scoring. Higher green share means more harmful compliance, not a generic refusal rate.

Capability retention

HumanEval: network-disabled execution, 164 tasks. Strict MMLU: eight-token cap, format-sensitive. Small differences are not proof of superiority.

Coding and harness checks

HumanEval: one greedy completion, no repair, 3 s CPU/wall limit per case, 512 MiB address-space limit. Editing checks are bounded fixtures, not SWE-bench.

Chat: throughput and interactivity

Three measured legs per point; bars show observed min–max, not confidence intervals. Native repeat has measured endpoints only.

Warm agent: throughput and interactivity

Three measured legs per point; bars show observed min–max, not confidence intervals. Native repeat has measured endpoints only.

Reasoning agent: throughput and response latency

Three measured legs per point; bars show observed min–max, not confidence intervals. Native repeat has measured endpoints only.

Cold agent context: concurrency costs

Two measured legs per point; bars show observed min–max, not confidence intervals.

Draft acceptance by target format

Three measured legs. Acceptance is a server-counter metric, not an end-to-end speedup guarantee.

Drafter comparison on the native target

Three measured legs. Drafting-off reasoning was not measured. Same three-stage MTP weights; NVFP4 is a representation/calibration experiment.

Long-input prefill

Three measurements per length. Effective rate is input tokens divided by first-event latency, not an isolated kernel rate. OS file cache was not standardized.

Storage and context

300K total context includes input + output. Thirty-two scheduler slots do not prove 32 simultaneous 300K sessions. Harness fixtures are bounded checks.

Inference settings

Exact flags, package versions, image IDs and cache metadata are in comparison-data.json. Main target quantization differs from the optional NVFP4 drafter.

How these comparisons were measured

These charts compare the same DeepSeek V4.1 Flash 1017 checkpoint family on four RTX PRO 6000 Blackwell 96GB GPUs. They include the original base and alternative edits where measured. Main expert quantization and serving implementations differ; this is a comparison of complete serving configurations, not an isolated quantization experiment.

Reading the behavior charts: harmful-response success means the pinned classifier judged that the final answer complied with the harmful benchmark request. It is not a generic refusal rate. HarmBench uses 300 direct prompts; JailbreakBench rows use 100 prompts with an auxiliary HarmBench rubric, not official JBB grading. The classifier passed 95/100 balanced validation controls. Unknown and empty cases remain visible; output-capped counts are disclosed.

Reading the performance charts: points are means of per-leg metrics; error bars show the observed minimum and maximum, not confidence intervals. Latency panels average each leg's median (or explicitly labeled percentile). Three legs were measured except cold-context waves, which used two. Natural output lengths and unrelated shared-GPU workloads vary. The white native-repeat markers are measured C1/C32 endpoints only.

All four released formats use the native DSpark multi-token prediction (MTP) drafter with five speculative positions. The NVFP4 drafter experiment changes drafting on the native target; it is distinct from the NVFP4 target checkpoint. Its gains are not assumed to transfer to EXL3. Both EXL3 implementations retain native MTP weights.

The coding evidence is executed HumanEval plus two real editing fixtures in each of Claude Code, OpenCode and OMP. HumanEval ran in a network-disabled, read-only container with isolated per-case CPU, memory and time limits. These are function-level coding and bounded integration checks. Repository-level SWE-bench has not been measured.

Native/MXFP4 is one mixed-format baseline. EXL3 bit rates refer to the main routed experts, not the whole model. Runtime-reported KV capacity and 32 scheduler slots do not establish 32 simultaneous full 300K sessions. Source measurements are from October 17–18, 2026.

Numeric data and inference settings

The text tables below contain the values plotted above. Download complete source measurements and exact runtime settings, including all individual legs, image IDs, package versions, server flags, cache metadata, classifier counts and source hashes. Plain-text tables are also available.

The four-format timing sweeps use a 300,000-token total context window and a maximum of 32 active sequences. Chat/warm/cold sweeps use temperature 0, top-p 1 and a 768-token output cap. Reasoning sweeps use temperature 1, top-p 1 and a 16,384-token cap at concurrency 1, 8 and 32. Prefill uses unique public-code prefixes and a short READY reply. Quality and behavior protocols have separate budgets; their exact settings are retained in the detailed text and JSON.

Recreate the figures after downloading the JSON and plotting script:

uv run --no-project --with matplotlib --with numpy python evaluation/plot-comparisons.py --bundle evaluation/visual-comparison.json --out comparison-figures

Original release notes

The original release notes below retain their historical wording. The four-format comparison is now complete; current comparison results, charts and inference settings are above.

DeepSeek V4.1 Flash 1017: measured native MXFP4 reference

This is the original mixed-precision checkpoint with pqhaz's published 38-tensor attention edit applied. Routed expert weights remain native MXFP4 and Engram tables remain FP8. “Native” and “MXFP4” describe this same reference, not two separate quantizations. We did not originate the edit or run the upstream Heretic optimizer.

This is an interim measured reference. Comparisons against another attention edit, NVFP4 and EXL3 are ongoing. It is not presented as the universally best method or as having zero refusals.

Source and measurements

Base revision: 1bae0e1088993b9b82de08f59de5be5a34e42170.
Patch repository: pqhaz/DeepSeek-V4.1-Flash-1017-heretic, revision 12d58ad0878b5008e80d720251adb969a0069412.
Patch SHA256: 16e63cd3e79c3590855aad340b5b9dc37f27562bf2d0a40f85268915da221ce2.

Local paired chat-mode scores: GSM8K 1276/1319, executed HumanEval 156/164, strict-format MMLU 11824/14042. Original base: 1279, 157 and 11920 respectively. These are local protocols, not official leaderboard scores. Temperature-zero output is not bitwise deterministic.

On four RTX PRO 6000 Blackwell Workstation GPUs, native drafting achieved mean aggregate reasoning throughput of 131.55, 425.19 and 830.06 output tokens/s at concurrency 1, 8 and 32. Three repeated legs used a warm 40K public-code prefix, natural stopping, temperature 1, top_p 1 and a 16384-token output cap. All nine legs passed integrity checks. Unrelated training and embedding workloads remained active.

The official HarmBench classifier passed 95/100 balanced human-majority controls. It scored direct-prompt behavior success on 287/300 edited outputs versus 30/300 base outputs. This measures behavior success, not a generic refusal rate. Full methodology, limitations and plots are in the benchmark dataset.

Run on four 96GB Blackwell GPUs

Use Linux, Docker with NVIDIA Container Toolkit, and sufficient disk for approximately 510.3GB of serialized tensors. The pinned runtime streams Engram tables from disk. The configured 300000-token context and 32 scheduler slots do not imply 32 simultaneous full-context sessions. Runtime cache metadata estimates approximately 15 full 300K contexts before workload overhead; that concurrency was not measured.

hf download Infatoshi/DeepSeek-V4.1-Flash-1017-MXFP4-uncensored --local-dir /data/deepseek-model
bash /data/deepseek-model/runtime/serve-native.sh /data/deepseek-model /data/deepseek-cache

The launcher never stops or replaces an existing container. Wait for http://127.0.0.1:8317/health to succeed. The local API model ID is deepseek-v41-local. Remote clients should forward loopback port 8317 over SSH. MODE=off disables speculative decoding for a matched comparison; the default uses DSpark with up to five proposed tokens.

bash /data/deepseek-model/runtime/harnesses/claude-ds.sh
bash /data/deepseek-model/runtime/harnesses/opencode-ds.sh
uv run --no-project python /data/deepseek-model/runtime/harnesses/configure-omp.py
bash /data/deepseek-model/runtime/harnesses/omp-ds.sh

Install the corresponding clients first. The OMP configuration helper preserves existing providers and refuses to overwrite an existing local-provider entry. The Claude wrapper retains normal instructions, hooks and plugins, selects bypassPermissions, and routes Agent-tool subagents to this same local model. Plain Claude commands launched through Bash regain the normally configured provider. No existing credentials are bundled.

The measured native runtime passed two editing tasks in each harness and a real Claude session reaching 260809 input tokens. That fixture does not establish arbitrary long-context task reliability. Incremental tool-driven growth passed; a single 250K-token initial stdin prompt failed Claude's estimator before reaching the API.

The exact packaged launcher and wrappers were also tested with this staged release as the only model mount. All six editing fixtures passed. The packaged Claude wrapper reached 260,205 input tokens with a reported 300,000-token window, 11 Read calls and no compaction, retaining the checked facts. evaluation/runtime-qualification.json records that full model and launcher check. The subsequent client-only OpenCode fix is recorded in evaluation/client-routing-qualification.json: an explicit CLI model selection prevents a configured agent default from redirecting the main session. A conflicting-agent fixture passed with the exact updated wrapper; server configuration and weight files are unchanged.

Weights retain the upstream MIT license. Runtime components retain their own licenses; included vLLM/B12X material is Apache-2.0. Full-file release hashes and provenance accompany the checkpoint. Raw benchmark requests, generated responses and private machine information are not included.

Additional installed-client long-context evidence

All three clients completed bounded tool-memory fixtures above 250K input tokens with the native model’s 300K server window and no compaction. Claude Code reached 260,205 input tokens using the exact packaged wrapper. Installed OpenCode and OMP aliases reached 281,477 and 255,419, retaining the checked early and final facts. The OpenCode test used stdin rather than a truncated file attachment. OMP required a verifier-only correction for filename:line-range read paths; its original receipt and session were preserved.

See evaluation/installed-harness-long-context.json for receipt hashes and scope. The OpenCode/OMP checks are installed-client evidence, not new exact-package runs. These fixtures do not certify every source line, arbitrary long-context reasoning, media support, or relative client speed. That evidence-only update changed no runtime, configuration or weight file. The later OMP client update is described below.

OMP context budgeting

The bundled omp-ds.sh loads omp-context.yml for that invocation. Automatic compaction and mid-turn checks remain enabled. It reserves 24,576 tokens: the 16,384-token output budget plus 8,192 tokens of headroom. With the 300,000-token window, the reserve-based trigger is 275,424 tokens. The same profile is used by the installed local alias. Global settings and other providers are unchanged.

OMP normally reserves 15% when this setting is unset. An earlier NVFP4 package fixture compacted at 269,638 input tokens under that policy and failed the qualification gate; the failed receipt was preserved. The revised native client configuration passed all six editing tasks. Its exact packaged OpenCode and OMP wrappers reached 281,805 and 259,066 input tokens, retained the checked facts, and completed without compaction or fixture changes. These remain bounded tool-memory tests.

evaluation/omp-context-qualification.json binds the previous and tested runtime hashes and the client results. Only the OMP wrapper and its new configuration file changed; model weights and server configuration did not. The prior Claude long-context result remains recorded separately. See OMP settings for the reserve and per-run overlay semantics.