license: mit
library_name: llama-modchip
inference: false
base_model: prem-research/DeepSeek-V4-Flash-0731-abliterated
base_model_relation: quantized
tags:
- gguf
- llama-modchip
- deepseek-v4
- abliterated
- uncensored
- mixture-of-experts
- cpu-inference
- quantization
- claude-code
- avx512
- avx2
- local-llm
pipeline_tag: text-generation
quantized_by: Synchopate
DeepSeek-V4-Flash-0731-Abliterated-Fractal-64GB-CPU-GGUF (284.33B in 56.93 GiB — Single-Pass Q6_K + Fractal-64)
[!IMPORTANT]
Custom Tensor Type (GGML_TYPE_Q2_0 = 42) — Do NOT use vanillaOllama,LM Studio,vLLM, or unpatchedllama.cpp:
Upstream engines do not define custom block-64Fractal-64(type 42) or thedeepseek4MLA / Hadamard / Sinkhorn-20 operators and will fail immediately withunknown tensor type 42(or40). Use the ISA-matched prebuilt binaries included in this repository (llama-server-modchip-linux-x86_64-avx512for Zen 4/5 & Ice Lake+, orllama-server-modchip-linux-x86_64-avx2for Haswell / Skylake / Zen 2/3 / Alder & Raptor Lake with zero%zmminstructions) or run./modchip setupfromsynchopate/llama-modchip.
Maximum-quality 64 GB CPU edition (61,129,364,000 bytes = 56.93 GiB / 61.13 GB across 3 shards) of DeepSeek-V4-Flash-0731-abliterated (284.33B total / 13.17B active parameters; 43 layers, 256 routed + 1 shared expert/layer, unpruned Top-6 routing). Quantized in a single pass directly from native BF16 / FP8_E4M3 / MXFP4 safetensors with zero intermediate GGUF re-quantization, upgrading all 130 dense 100%-active bottleneck matrices (token_embd, output, and all 43 layers of attn_q_a, ffn_gate_shexp, ffn_up_shexp) to Q6_K (6.5625 bpw) while preserving 100% of MLA attention in Fractal-64 (3.75 bpw) and all 256 routed MoE experts via @antirez's 1.5M-token importance matrix.
[!WARNING]
Abliterated / Uncensored Checkpoint — Safety, Refusal Removal, & Responsible Use Disclaimer:
- What "Abliterated" Means: This model is quantized from
prem-research/DeepSeek-V4-Flash-0731-abliterated, which applies orthogonal refusal-direction projection to the residual-stream weights ofdeepseek-ai/DeepSeek-V4-Flash-0731to remove alignment-triggered refusals without retraining.- Disabled Safety Guardrails: Built-in refusal filters present in the standard
DeepSeek-V4-Flash-0731checkpoint are inactive. The model will answer dual-use technical queries (reverse engineering, exploit mechanics, offensive/defensive security, red-team auditing) directly and may generate sensitive, harmful, offensive, or inaccurate content if prompted.- Intended Scope & Deployment Restrictions: Provided strictly for authorized security research, CTF/vulnerability auditing, academic systems evaluation, and local offline development. Do not expose this model on public, multi-user, or unauthenticated endpoints without independent external safety moderation. Users bear sole legal and ethical responsibility for ensuring their use complies with all applicable laws and regulations.
- Standard Aligned Alternative: If your deployment requires standard upstream refusal alignment, use
Synchopate/DeepSeek-V4-Flash-0731-VQ3-64GB-CPU-GGUF(56.40 GiB) orSynchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF(56.57 GiB).
Head-to-Head Quality & 64 GB CPU Benchmark (v2.5.0)
Measured on a 16-Core AMD Ryzen 9 9955HX (64 GB DDR5-5600, 0 GPUs, Linux 6.8, Transparent HugePages):
| Metric / Suite | DeepSeek-V4-Fractal (56.57 GiB) |
DeepSeek-V4-VQ3 (56.40 GiB) |
Abliterated-Fractal-MaxQ (56.93 GiB) |
Delta vs. Fractal Ref |
|---|---|---|---|---|
| GGUF File Size (3 Shards) | 56.57 GiB |
56.40 GiB |
56.93 GiB (61,129,364,000 B) |
+0.36 GiB (130 tensors in Q6_K) |
Peak Resident RAM (VmRSS, -c 32768) |
58.20 GiB |
58.31 GiB |
59.13 GiB (~3.4 GiB OS headroom) |
Fits in 64 GB DDR5 (0 swap) |
Continuous llama-perplexity (4,096 tok) |
1.8284 ± 0.07038 |
1.8258 ± 0.06978 |
1.7597 ± 0.06534 |
-0.0687 (-3.76% lower PPL) |
1. Agentic Tool-Calling (30 pts) |
24.0 / 30.0 |
26.0 / 30.0 |
30.0 / 30.0 |
+6.0 (100% JSON schema match) |
2. Executable Code Gen (30 pts) |
20.0 / 30.0 |
20.0 / 30.0 |
20.0 / 30.0 |
+0.0 |
3. Long-Context Needle & Math (20 pts) |
20.0 / 20.0 |
20.0 / 20.0 |
20.0 / 20.0 |
+0.0 (3,038-tok canary + KV math) |
Functional Quality Gate Subtotal (80 pts) |
64.0 / 80.0 |
66.0 / 80.0 |
70.0 / 80.0 |
109.4% of Fractal reference |
Extended Hard-60 Coding/Math Suite (60 pts) |
20.0 / 60.0 |
40.0 / 60.0 |
50.0 / 60.0 |
+30.0 pts (2.5x Fractal score) |
Technical Security Refusal Audit (8 prompts) |
8 / 8 |
8 / 8 |
8 / 8 |
Zero alignment refusals on CTF/audit tasks |
Cold / Warm Prompt Prefill (tok/s) |
37.8–44.0 / 44.36 |
41.1–44.0 / 44.36 |
37.8–43.5 cold / 44.0 warm |
30.7 tok/s cold at 3,038 tokens |
Decode (N=1 Unpruned Top-6 / Speculative) |
9.55 / 11.5–21.2 |
9.05 / 19.4–25.3 |
8.75 tok/s (N=1) / 11.2–21.0 spec |
-0.8 tok/s from Q6_K output.weight |
Single-Pass Surgical Quantization Architecture (Zero Double-Quantization)
Unlike multi-step GGUF transcodings (FP8/MXFP4 -> Q8_0/Q5_K -> target), every quantized tensor in this checkpoint was encoded in one pass directly from the native upstream safetensors:
- 130 Dense 100%-Active Bottlenecks ->
Q6_K(6.5625 bpw, Direct fromBF16/FP8_E4M3):token_embd.weight[4096, 129280]&output.weight[4096, 129280](BF16 -> Q6_K).- All 43 layers of
blk.0..42.attn_q_a.weight,blk.0..42.ffn_gate_shexp.weight, andblk.0..42.ffn_up_shexp.weight(FP8_E4M3 * scale -> Q6_K), plusblk.0..42.ffn_down_shexp.weight(42 × Q6_K+1 × Q8_0). Because shared experts and query-LoRA down-projections execute on 100% of tokens, eliminatingQ4_K/Q5_K/VQ3compression noise here drives the-0.0687Perplexity drop and+30-pointHard-60gain.
- 317 MLA Attention & Lightning Indexer Projections ->
Fractal-64(GGML_TYPE_Q2_0 = 42,3.75 bpw, Direct fromFP8_E4M3):- Encodes
attn_q_b,attn_output_a,attn_output_b,attn_kv,attn_compressor_*, andindexer_*directly fromFP8_E4M3 * scaleinto 64-weightblock_q2_0dyadic blocks (17.75 dBSQNR,99.15%Cosine Similarity), compatible with x86-64 AVX-512, x86-64 AVX2, and Apple Silicon Metal (M5 Max 64 GB).
- Encodes
- 129 3D Routed-MoE Tensors (
256Experts/Layer) -> Single-PassIQ2_XXS/IQ1_S/IQ1_Mwith1.5M-Tokenimatrix:- Dequantized in RAM from native
MXFP4(E2M1+UE8M0) and quantized in a single pass weighted by Salvatore Sanfilippo's (@antirez)1.5M-tokenrouted-moe-ds4-1p5m.datcalibration matrix (IQ2_XXSon anchor layers{0, 1, 26, 28, 41, 42},IQ1_Mon mid-sensitivitydown_exps,IQ1_Sbulk).
- Dequantized in RAM from native
- Norms, Router Gates, Token-ID Hash Tables (
tid2eid), & Sinkhorn-20 Hyper-Connections: Kept in fullF32/BF16/I32.
Quickstart
- 64 GB OS Headroom (
-c 32768vs-c 262144): At-c 32768, peakVmRSSis59.13 GiB, leaving~3.4 GiBof OS headroom inside~62.5 GiBusable Linux RAM on a 64 GB machine (close heavy browser sessions on GUI desktops, or enablezram). Use-c 262144(256Ktokens) on dedicated headless 64 GB nodes withzram/swap or on96 GB+workstations. - Matching MLA KV Cache Types: Always pass matching K and V cache types (
-ctk q8_0 -ctv q8_0or-ctk q4_0 -ctv q4_0).
Option A: Automated llama-modchip Installer (Default DeepSeek on 64 GB+ Hosts)
git clone https://github.com/synchopate/llama-modchip.git
cd llama-modchip
./modchip setup # Interactive selector defaults to this model on 64GB+ machines
./modchip chat
Option B: Direct Run with Bundled ISA-Matched Linux x86_64 Binary (-avx512 or -avx2)
hf download Synchopate/DeepSeek-V4-Flash-0731-Abliterated-Fractal-64GB-CPU-GGUF --local-dir ./dsv4-ablit
cd ./dsv4-ablit
# Auto-selects -avx512 (Zen 4/5, Ice Lake+) or -avx2 (Haswell, Zen 2/3, Alder/Raptor Lake — zero SIGILL):
BIN="./llama-server-modchip-linux-x86_64-$(grep -q 'avx512_vbmi' /proc/cpuinfo && echo avx512 || echo avx2)"
chmod +x "$BIN"
"$BIN" \
-m DeepSeek-V4-Flash-0731-Abliterated-Fractal-00001-of-00003.gguf \
-c 32768 -t 16 -tb 16 -b 2048 -ub 2048 -fa on -ctk q8_0 -ctv q8_0 --port 8080 --jinja
Credits & Acknowledgments
This release builds directly on the open engineering of three upstream teams and authors:
- Salvatore Sanfilippo (
@antirez) —antirez/deepseek-v4-gguf: Special thanks for his single-pass DeepSeek-V4 MoE GGUF work and for computing and sharing the1.5M-tokenDeepSeek-V4-Flash-chat-v2-routed-moe-ds4-1p5m.datimportance matrix that guides every routed-expert quantization block in this checkpoint. - Prem Research (
prem-research) —prem-research/DeepSeek-V4-Flash-0731-abliterated: For producing and releasing the full-precision (BF16/FP8_E4M3/MXFP4) abliterated master weights. - DeepSeek AI (
deepseek-ai) —deepseek-ai/DeepSeek-V4-Flash-0731: For training and releasing the foundational284.33BDeepSeek-V4-Flashmodel under the MIT License.