license: mit
base_model:

• dealignai/DeepSeek-V4.1-Flash-UNCENSORED-FP8
• deepseek-ai/DeepSeek-V4.1-Flash
  base_model_relation: quantized
  tags:
• deepseek
• deepseek-v4.1
• moe
• quantized
• 2-bit
• ternary
• mixed-precision
• gptq
• uncensored
• strata
• consumer-gpu

Upload in progress (started 2026-10-12): the 16 model parts arrive over about a day at home upload speed.
SHA256SUMS already lists every part; reassemble.sh refuses to join until all of them are present and intact.

Experimental research release from a single developer, measured on one PC.
This file is not a llama.cpp / Ollama / LM Studio GGUF. It uses per-expert quant types those runtimes don't load.
Use it with Strata-DSV4.1.

What this is

DeepSeek-V4.1-Flash has 40 layers with 384 routed experts each
(top-6 per token), a shared expert, Hyper-Connections, CSA sparse attention, and an Engram n-gram memory. Its
15,360 routed experts hold 543.6B weights; even at their native FP4 they're far bigger than any consumer PC's RAM.

This repository holds a calibrated mixed-tier quantization of
dealignai's abliterated checkpoint, in which
every expert's gate/up and down projections get their own format:

The dense half (attention, shared expert, output head) is Q8_0. The embedding, compressor and indexer stay F16,
and norms, router and mHC stay F32.

The total is budget-neutral: exactly the bytes of a uniform 2.25-bit (all-Q2_0) quant, but spent where calibration
says they matter. Nearly half the expert weights are ternary, which pays for 3-bit on the experts that need it. On the
calibration proxy, the mix has 17% less output error than uniform Q2_0 at the same size.

"GSQ" names the method family this follows: the format grids and the plan come from
GSQ (Gumbel-Softmax Quantization) and
RCO (budget-exact allocation) by IST Austria's DAS Lab. This release is the
GPTQ-initialized stage of that pipeline with our own "RCO-lite" allocation. GSQ's Gumbel-softmax refinement
hasn't been applied yet; see Not done yet below.

Why

We wanted a frontier-class open model running locally, on hardware people actually own, without slicing it down to a
distilled toy. The design question was where the bits matter. A MoE spends most of its parameters on experts that
each token touches only rarely, so we spend the bit budget where the calibration says it counts, and run the rest
as cheaply as the quality numbers allow.

The engine then treats the experts as a three-tier cache:

How it was made

The whole pipeline is open source in
Strata-DSV4.1/tools/v41. It runs on a single
RTX 4090.

0. Weight-only metrics can't rank these experts. V4.1's routed experts were trained in FP4, so all 384 experts of a
layer share the same per-32 weight distribution. Every expert in layer 0 has the same round-to-nearest error to three
decimals (ternary 0.443 / Q2 0.378 / Q3 0.183 relative). Which experts deserve more bits can only come from
activations: what the experts actually compute on real text.

1. Calibration corpus (agentic-first). Real coding-agent session transcripts (secrets scrubbed value by value, tool
results clipped) plus antirez/ds4's 4,690 imatrix prompts, all re-rendered with DeepSeek's own V4.1 chat encoder (DSML
tool-call format). That's 128 sequences × 2,048 tokens, with about a quarter of each domain held out: 98 sequences
fit the quantizer, and 30 only ever measure it.

2. A layer-streaming teacher forward. DeepSeek's own reference model.py runs one layer at a time straight
from the HF shards, keeping the hidden state on disk between layers. That makes the teacher the real model, not an
approximation. Each layer's MoE runs expert-major: every FP4 expert is dequantized once and sees all of its routed
tokens.

3. Per-expert GPTQ at three exact grids. For every expert, gate+up are fitted on the Hessian of its input, and down
on the Hessian of its clamped-SwiGLU × routing-weight input. Scales are chosen before GPTQ by a diag(H)-weighted
search (11 clip ratios × both signs) and rounded to what each format stores, so every result is exactly representable
in its GGUF type. GPTQ (damp 1.0) then picks the integers. It beat round-to-nearest in every group (error ratios
0.58 ternary / 0.70 Q2 / 0.74 Q3). Each expert's error is measured on the held-out sequences only, as output error
with the routing weight included.

4. Budget-exact allocation ("RCO-lite"). The cost of a choice is held-out output error divided by the layer's
routed output energy, so every layer counts equally; the first two and last three layers count double. Gate+up and
down are allocated separately. The budget is exactly the all-Q2_0 byte count, solved by Lagrangian relaxation
(bisection on λ) plus a greedy pass to spend the remainder. Guard rails: at most 50% ternary per layer, and experts
with fewer than 8 held-out tokens are floored at Q2_0, so an expert can't drift to ternary without evidence.

5. Sequential quantize pass. A second layer-by-layer chain quantizes every expert at its chosen format. Each layer
is fitted on the outputs of the already-quantized layers below it, so later layers adapt to the errors earlier
ones actually make.

6. Bit-exact export. Block encoders write the GGUF bytes directly and are checked bit for bit against ggml's
dequantizers and the PTQ1_0 reference. Ternary packs 5 trits per byte (28 bytes per 128 weights), and Strata runs it
natively on the CPU and in its grouped-expert CUDA kernels.

Where the bits went

Measured on the held-out calibration tokens; the relative output error is each layer's quantized routed output against
the FP4 original, in the final sequential pass.

Not done yet

• GSQ refinement: Gumbel-softmax optimization of the grid assignments at each expert's chosen format. This is
  the step that closes most of the gap to vector quantization in the GSQ paper. It slots in where GPTQ runs and
  writes the same format.
• A Hadamard rotation for the ternary tier.
• The full RCO manifold search: the allocation above is its Lagrangian "lite" version.

Quality

Perplexity on the 49,146 held-out calibration tokens (never used for fitting): the quantized model vs the original
checkpoint ("teacher"):

Top-1 token agreement: 0.674 (GSQ) vs 0.701 (teacher). In the engine, a 4096-token code prompt gives ppl 2.380 and top-1 0.822.

Prompt processing is exact: every prefill speed-up in the engine keeps the last-token logits byte-identical.
Decode uses two selection-only routing biases by default: a bonus for VRAM-resident experts and a penalty for
NVMe ones. They trade a little perplexity for speed (code 2.38 → 2.87). The mixing weights stay exact, every expert
stays reachable, and --route-bias 0 --route-bias-file 0 gives fully exact routing, which is much slower.

Speed (RTX 4090 24 GB · Ryzen 7 5700X · 90 GB DDR4 · NVMe)

How each number was reached, step by step:
V41_PROGRESS.md.

Files

Reassemble

hf download IndrasMirror/DeepSeek-V4.1-Flash-GSQ-Uncensored --local-dir V41-GSQ
bash V41-GSQ/reassemble.sh V41-GSQ                 # needs ~163 GB free on top of the parts
bash V41-GSQ/reassemble.sh V41-GSQ --delete-parts  # or: delete each part once appended (~163 GB total)

Running it

You also need the Engram tables (202.8 GB). They aren't in this repo because they're unchanged FP8 rows from the
original checkpoint, so there is no point hosting a third copy. Download the two Engram shards
(model-00047-of-00048.safetensors and model-00048-of-00048.safetensors, 101.5 GB each) plus
model.safetensors.index.json and config.json from
dealignai/DeepSeek-V4.1-Flash-UNCENSORED-FP8.
The abliteration left Engram byte-identical to the base model, and
deepseek-ai/DeepSeek-V4.1-Flash has the same two shards,
so either source works. Then extract them to fast drives (one table per drive is ideal):

hf download dealignai/DeepSeek-V4.1-Flash-UNCENSORED-FP8 model-00047-of-00048.safetensors \
   model-00048-of-00048.safetensors model.safetensors.index.json config.json --local-dir FP8-engram

git clone https://github.com/Indras-Mirror/Strata-DSV4.1 && cd Strata-DSV4.1
# build (CUDA, sm_89 shown) - see the repo README for the full line
cmake -S . -B build-ds4-gpu -G Ninja -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_GGML_CUDA=ON \
      -DSTRATA_MMQ_KQUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=89 -DSTRATA_GGML_DIR=$PWD/third_party/llama.cpp
ninja -C build-ds4-gpu ds41_generate
python3 tools/v41/extract_engram.py FP8-engram --to layers.1=/fast/a --to layers.14=/fast/b
STRATA_NO_LARGEPAGES=1 build-ds4-gpu/ds41_generate -m V41-GSQ.gguf --ids-file prompt.i32 -n 128 \
     --backend cuda --experts gpu --engram "/fast/a:/fast/b" --rank rank.bin

An OpenAI- and Anthropic-compatible server with tool calls (tools/v41/serve/) is in the repo too.

Minimum hardware: an NVIDIA GPU with 24 GB, ~90 GB RAM (the RAM arena is 60 GiB by default; --arena-gib
lowers it, at some speed cost), and fast NVMe with room for the model plus the Engram tables (~366 GB total).

Credits

• DeepSeek for DeepSeek-V4.1-Flash (MIT).
• dealignai for the abliterated FP8 checkpoint this is quantized from (MIT).
  This release inherits that abliteration: the model has no refusal behaviour, and you are responsible for how you
  use it.
• GSQ (Dadgarnia, Tabesh, Nikdan, Helcig, Kurtić, Kleinegger, Alistarh) and
  RCO (Helcig, Alistarh), IST Austria DAS Lab, for the quantization grids
  and the budget-exact allocation idea this pipeline follows.
• antirez/ds4 for its 4,690 imatrix calibration prompts (re-rendered for V4.1).
• llama-tq for the PTQ1_0 ternary block format (type 143) and its reference kernels (MIT).
• Strata (Niko1221), the tiered MoE engine this work builds on;
  ggml / llama.cpp for the compute backend.
• Quantization pipeline, engine port and measurements: Indra's Mirror (Strata-DSV4.1).