license: mit
language:

• en
  library_name: gguf
  pipeline_tag: text-generation
  base_model: cyberneurova/CyberNeurova-DeepSeek-V4-Flash-abliterated-GGUF
  base_model_relation: quantized
  tags:
• gguf
• llama.cpp
• deepseek
• deepseek-v4
• mixture-of-experts
• abliterated
• uncensored
• imatrix
• quantized
• apple-silicon
• metal
• macbook-pro
• m5-max
• 128gb
  model_name: CyberNeurova DeepSeek V4 Flash Abliterated v2 IQ3_XXS-AS F32Router

CyberNeurova DeepSeek V4 Flash Abliterated v2 — IQ3_XXS-AS F32Router GGUF

Tailor-made for the 128 GB M5 Max MacBook Pro. This quant was
designed, built, and benchmarked on that exact Apple-silicon machine. Its
95.48 GiB weights fit inside the tested Mac's 107.5 GiB Metal-visible
allocation pool while leaving about 12 GiB for runtime state and context.

This is a custom importance-matrix quantization of
cyberneurova/CyberNeurova-DeepSeek-V4-Flash-abliterated-GGUF.
It specifically targets the 128 GB unified-memory M5 Max MacBook Pro and
similar machines where the Q8 release is too large but preserving more quality
than the smallest quants is worthwhile.

This repository only contributes the quantization. CyberNeurova created the
abliterated v2 weights; DeepSeek created the original V4 Flash model.

Artifact

llama-gguf-hash tensor-data SHA-256:
12e66621bd390b79904463cf3cdb5b288bf7265352cf1b66740e211c8f39c7b3.

Quantization layout

The installed llama-quantize 10090 dry run reported:

model size = 288244.36 MiB (8.50 BPW)
quant size =  97767.25 MiB (2.88 BPW)
imatrix entries = 129

The tensor layout was reconstructed by comparing the CyberNeurova Q8 GGUF
with bullerwins/DeepSeek-V4-Flash.IQ3_XXS-AS.gguf. Both contain the same
1,328 tensor names and dimensions. The published AS reference stores its 43
router-input tensors as BF16. Current llama.cpp excludes those tensors from
re-quantization, so this build preserves CyberNeurova's F32 routers instead.
That costs about 86 MiB and retains higher precision.

The complete per-tensor mapping is included at
recipe/iq3_xxs_as.tensor-types.txt.

Inputs and reproducibility

The importance matrix is the published Jedi chat-v2 routed-MoE matrix used for
Audrey Tang's aligned CyberNeurova re-quant.

The build used llama.cpp version 10090 (7347430f4) and:

llama-quantize \
  --allow-requantize \
  --imatrix DeepSeek-V4-Flash-chat-v2-routed-moe-ds4-aligned.dat \
  --tensor-type-file recipe/iq3_xxs_as.tensor-types.txt \
  --override-kv \
  general.name=str:CyberNeurova-DeepSeek-V4-Flash-Abliterated-v2-IQ3_XXS-AS-F32Router \
  cyberneurova-DeepSeek-V4-Flash-abliterated-Q8_0.gguf \
  CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-F32Router.gguf \
  IQ3_XXS

Validation checked every tensor name, dimension, and requested tensor type.
It also hash-compared every F32, I32, and Q8_0 tensor expected to remain
byte-identical to the CyberNeurova Q8 source.

Download

hf download \
  pyrodog/CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-GGUF \
  CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-F32Router.gguf \
  chat_template.jinja \
  --local-dir .

Run with llama.cpp

The GGUF does not embed a chat template. The companion
chat_template.jinja is the exact template extracted
from a clean DeepSeek V4 Flash reference GGUF. Pass it explicitly; otherwise
llama.cpp may fall back to ChatML and produce misleadingly poor behavior.

The following is the tested 32K-context Apple Metal profile:

llama-server \
  -m CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-F32Router.gguf \
  -ngl 999 \
  -fa on \
  -ctk q8_0 \
  -ctv q8_0 \
  -c 32768 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning-format deepseek \
  --reasoning on

The model was tested on a 128 GB M5 Max. The 95.48 GiB weight file left about
12 GiB of that machine's 107.5 GiB Metal-visible allocation pool for runtime
state and context. Actual usable memory varies by system and background load.

Local A/B measurements

The custom quant was compared with Audrey Tang's 90.89 GiB mixed quant under
the same mainline llama.cpp Metal runtime. Both used all GPU layers, flash
attention, Q8 KV caches, a 32,768-token context, temperature 0, and seed
424242.

For 4.59 GiB more storage, this quant was 42.05% faster on the 512-token
prompt, 10.41% faster on the 8,192-token prompt, and 12.09% faster in
generation. Its measured perplexity was 9.90% lower.

These are controlled measurements on one machine, not broad independent
benchmarks. The 12-task smoke suite is particularly small; a formatting-only
miss on each side makes the semantically lenient score a 7/12 tie. The speed
and perplexity results are the stronger reasons to prefer this quant.

Audrey's specialized ds4 runtime can execute the smaller mixed quant's
IQ2/Q4 experts much faster, but does not implement this model's IQ3 expert
dispatch. Use this quant for the measured fidelity/performance balance under
mainline llama.cpp; use the smaller mixed quant with ds4 when specialized
decode speed is the priority.

Intended use and limitations

This is an experimental research artifact for local inference, defensive
security research, academic study of refusal mechanisms, red-team evaluation,
and controlled coding-agent experiments.

The underlying model has had canonical refusal behavior deliberately reduced.
That does not make its outputs correct or safe. It can produce harmful,
insecure, biased, or fabricated content. Review generated code before running
it, use sandboxing for tools, and do not use the model to facilitate illegal
activity or harm.

Additional limitations:

• Long-context behavior beyond 32K was not validated for this quant.
• The benchmark covers one Apple-silicon machine and one llama.cpp build.
• Quantization adds error on top of the abliterated source weights.
• The GGUF requires the external chat template included in this repository.
• Mainline and specialized DeepSeek V4 runtimes are evolving quickly.

License and acknowledgements

MIT, inherited from DeepSeek V4 Flash and the CyberNeurova release. See
LICENSE.

Thanks to:

• DeepSeek-AI for V4 Flash
• CyberNeurova for the v2 abliterated weights
• Audrey Tang for the aligned comparison quant and runtime work
• Jedi for the aligned importance matrix
• bullerwins for the IQ3_XXS-AS tensor-layout reference
• llama.cpp contributors for GGUF tooling and inference support