license: mit
library_name: gguf
pipeline_tag: text-generation
base_model: apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8
base_model_relation: quantized
quantized_by: apetersson
tags:

• gguf
• quantized
• deepseek
• deepseek-v4
• deepseek-v4-flash
• moe
• mixture-of-experts
• abliterated
• mxfp4
• iq2_xxs
• q2_k
• ds4
• dspark
• apple-silicon
• metal

DeepSeek V4 Flash Abliterated — DS4 Quality128

Exact MXFP4 experts, maximum resident quality.

Runtime compatibility: the target GGUF runs with both DS4 and
llama.cpp on Metal. For DSpark speculative decoding, use the companion
whose filename identifies the runtime: DSpark-support for DS4 or
llamacpp-DSpark-support for a llama.cpp build with DeepSeek V4 DSpark
support.

This is a quality-first GGUF package built with the DS4 Quality128 quantization
policy. It is designed to keep DeepSeek V4 Flash resident on a 128 GB M1 Ultra
while preserving the most sensitive routed experts in their exact native
MXFP4 representation. DS4 in the package name identifies the quantization
profile. Use the same target GGUF with either runtime; only the optional DSpark
companions are runtime-specific.

The model is quantized directly from the abliterated FP8 checkpoint
apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8.
The abliteration affects only 36 attention wo_b tensors; routed-expert codes
and scales are unchanged from that checkpoint.

Artifacts

The companions are runtime-specific alternatives. For DSpark, load only the
companion matching the runtime. Keeping all three GGUFs on disk uses
117,426,960,352 bytes (109.3624 GiB).

Quantization profile

• Exact native MXFP4 gate/up/down routed experts on layers
  10, 14, 30, 34, 37, 38, 39, 40, 41, 42.
• IQ2_XXS gate/up and Q2_K down routed experts on the other 33 MoE layers.
• Q8 attention, shared-expert and output paths.
• F16 protected indexer and auxiliary tensors.
• The routed-expert imatrix applies only to genuinely requantized IQ2/Q2
  tensors; preserved MXFP4 needs no imatrix.
• DSpark support uses IQ2_XXS gate/up and exact native MXFP4 down projections
  for target layers 40, 41 and 42.

Target GGUF type histogram (1,328 tensors): F32 492, F16 359, I32 3, Q8_0 345,
IQ2_XXS 66, Q2_K 33 and MXFP4 30. DSpark support histogram (81 tensors): F32
34, F16 7, Q8_0 31, IQ2_XXS 6 and MXFP4 3.

Runtime compatibility

The target GGUF is shared by both runtimes. The two companions use different
DSpark schemas and are not interchangeable:

Use the target GGUF directly for target-only inference in either runtime. For
DSpark speculative decoding, pair it with the companion listed for that
runtime. The DS4 companion requires
a recent ds4 version from the main branch,
with its DSpark generation support tracked in
antirez/ds4#642.

• Quantizer SHA-256: f0a381f4ada808ea2afa740d964354fa327fc1235ba7cebf50874eb89fb97ac5
• Runtime SHA-256: 2aaf20469b9918d6d6ab8787a02811c11228547cd979787879a03dba8a9e7824

The DS4 companion requires the recent ds4 main-branch version above. Stock
llama.cpp b10210 supports target-only inference but cannot use either
companion. llama.cpp DSpark requires the dflash companion and a build that
contains the DeepSeek V4 DSpark changes described below.

Validated llama.cpp configurations

Target-only with b10210

Target-only validation used a 128 GB M1 Ultra and stock Homebrew llama.cpp
build b10210 (000547513, 2026-07-31). The configuration used full Metal
offload, a 4,096-token context, 256-token batch and ubatch, Flash Attention, no
warmup and greedy decoding. A deterministic arithmetic probe returned the
correct answer, with preliminary measurements of 15.8 prompt tokens/s and 8.0
generation tokens/s. These figures are a single short smoke test, not a
sustained benchmark. This build supports target-only inference for this
package; neither companion is compatible with it.

DSpark with fffbcbdb

llama.cpp DSpark requires DeepSeek V4 MTP/DSpark, separate DSpark conversion,
sidecar discovery and Metal hyper-connection support. Validation used upstream
commit fffbcbdb9d5e56105a8842867a59bb9736520ca8 from 2026-08-02. Relevant
changes include
#25784,
#26458 and
#26459.

The llama.cpp sidecar contract uses architecture dflash, dflash.* metadata,
target tokenizer/model metadata and standardized tensor names such as blk.*,
markov_w1.weight and conf_proj.weight. The DS4 companion uses the DS4-native
deepseek4-dspark, dspark.* and mtp.* layout.

The llamacpp-DSpark-support.gguf companion is a container-only repack of the
DS4 companion. Its 81 tensor descriptors use llama.cpp names, and its header
contains the target tokenizer and standardized dflash metadata. It preserves
the complete 7,297,731,680-byte tensor-data region byte-for-byte without
requantization. The source and output payload SHA-256 is:

befbdb4a0f7e6b2626cfa9af0bc1560261a7a3235e0e6670b4f0c8df032a2e74

repack_llamacpp_dspark.py reproduces the conversion (SHA-256
9ffa5aedd0fa83b74846ff164aabd82097ee60e4610231493fe1ae5a702efb99)
without changing model weights or the DSpark quantization policy.

The validation configuration used a 128 GB M1 Ultra, full Metal offload, a
4,096-token context, 128-token batch and ubatch, Flash Attention, no warmup,
greedy decoding and a maximum DSpark block of five tokens. A short arithmetic
probe returned 42. A longer deterministic sequence produced the integers 1
through 40 correctly and reported:

These figures are a short correctness and compatibility probe, not a sustained
or cross-runtime benchmark. The tested build emits a nonfatal warning that the
Lightning Indexer for layer 2 is assigned to CPU and disabled; the warning does
not prevent DSpark from loading, drafting or accepting tokens.

Launch examples

llama.cpp b10210 target-only inference:

llama-cli \
  --model DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf \
  --ctx-size 4096 \
  --batch-size 256 --ubatch-size 256 \
  --gpu-layers 999 --flash-attn on \
  --reasoning off --reasoning-budget 0

DS4 target-only inference:

ds4 --metal \
  -m DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf

DS4 inference with its DSpark companion:

ds4 --metal \
  -m DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf \
  --mtp DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128-DSpark-support.gguf \
  --dspark --temp 0

llama.cpp DSpark inference with commit fffbcbdb:

llama-cli \
  --model DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf \
  --spec-type draft-dspark \
  --spec-draft-model \
    DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128-llamacpp-DSpark-support.gguf \
  --spec-draft-n-max 5 \
  --ctx-size 4096 \
  --batch-size 128 --ubatch-size 128 \
  --gpu-layers 999 --spec-draft-ngl 999 \
  --flash-attn on \
  --temp 0 --reasoning off --reasoning-budget 0

This command requires commit fffbcbdb or another build containing the
2026-08-02 DeepSeek V4 DSpark changes. Homebrew b10210 supports only the
target-only command above.

DS4 one-million-token context

The following residency estimates and --prefill-chunk recommendations apply
only to DS4; one-million-token context is unvalidated with llama.cpp.
--ctx 1048576 counts prompt and completion together. The safest
maximum-quality resident mode omits DSpark and uses a 2,048-token prefill
chunk:

ds4 --metal \
  -m DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf \
  --ctx 1048576 --prefill-chunk 2048

Estimated residency is at most 110.30 GiB, leaving at least 11.30 GiB
below Metal's approximately 121.60 GiB recommended working set. Omitting
DSpark does not reduce target-model quality; it only forgoes speculative decode.

Enable DSpark with the smaller chunk only after confirming peak memory on the
host:

ds4 --metal \
  -m DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128.gguf \
  --ctx 1048576 --prefill-chunk 1024 \
  --mtp DeepSeek-V4-Flash-0731-Abliterated-DS4-Quality128-DSpark-support.gguf \
  --dspark --temp 0

That mode is estimated at 114.02 GiB, about 7.58 GiB below the recommended
working set. DSpark with chunk 4096 is estimated at 123.24 GiB and is not a
reliably resident configuration.

Verification and provenance

The package passed strict target and DSpark planning, exact size/type/name-set
checks, strict imatrix coverage, source validation, and byte reproduction for
all 30 target plus three DSpark MXFP4 tensors. The llama.cpp companion passed its
81-tensor name contract, metadata contract, full payload size check,
byte-identical payload SHA-256 check and a real target-plus-draft inference
test. SHA256SUMS binds all three GGUFs, the repack script and the
documentation. See:

• BUILD_MANIFEST.json for pinned source shards, tools,
  commands and publication metadata.
• PROVENANCE.md for the compact lineage record.
• BUILD_PLAN.md for the completed build gates and benchmark
  matrix.
• ds4-upstream-issues.md for remaining DS4 runtime
  and memory improvements.
• repack_llamacpp_dspark.py for the reproducible,
  dependency-free llama.cpp sidecar repack.

The sibling MLX package
DeepSeek-V4-Flash-0731-Abliterated-MLX-Quality128-suboptimal is available for
speed comparison; it is not the canonical quality artifact.

Treat the native MXFP4 Metal kernels and mixed DSpark path as experimental
relative to Q4_K. Benchmark correctness and throughput against DS4 v1 and the
MLX comparator before selecting an everyday launch configuration.