license: other
license_name: kimi-k3
license_link: https://huggingface.co/moonshotai/Kimi-K3
base_model: BlackfrostAI/KIMI-K3-DERISKED-MXFP4
base_model_relation: quantized
tags:

• kimi-k3
• derisked
• gguf
• q2_k
• moe
• llama.cpp
• single-node
• experimental
• gated
• research
• security-research
• red-teaming
• adversarial-testing
  pipeline_tag: text-generation

Why this model exists

Full Kimi K3 in GGUF is ~1.5 TB. This is the same de-risked checkpoint requantized to Q2_K so it fits a single 8×B200 node with room for KV — with the refusal surface already reduced at the weight level in the parent.

The single-node GGUF footprint is the product; the refusal behaviour change is inherited from the parent and is the contract.

Specifications

What "Q2_K" costs here

The experts were already MXFP4 4-bit QAT in the source checkpoint. This is therefore a
requantization of already-quantized weights, not a quantization of BF16. That is a different
and harsher operation than a normal Q2_K build, and its quality relative to the MXFP4 parent
has not been measured.

Blackfrost's own quant ladder for K3 starts at IQ4_XS for exactly this reason. This Q2_K exists to prove single-node GGUF viability, not as a recommended quality point.

Full-precision and higher-quality builds

This Q2_K is the openly-distributed variant. As set out above, it is a requantization of
already-4-bit-QAT experts and is not the quality point Blackfrost recommends for production work.

Full-precision and higher-quality builds of this checkpoint are available for purchase under a
separate licence agreement, including:

Access to these builds is granted on purchase. Their repositories are gated, and approval
follows a completed licence agreement — the gate is the transaction. Requesting access to a
premium repository without a licence in place will not be approved.

Purchase link coming soon. Until then, @Blackfrost_AI DMs are
the fastest route to a human.

Lineage

On refusal behaviour: this checkpoint inherits the parent's deliberately reduced refusal surface. It is a Blackfrost de-risked model. Do not evaluate or rate-limit it as if it were a safety-stock derivative of upstream Kimi K3.

Measured behaviour

Verified on 8× NVIDIA B200, llama.cpp PR #26185 branch @ cf67f0d, llama-server:

Quality — not yet measured

Requantizing already-QAT experts is a capability trade, and a card that ships a Q2_K of a 4-bit-QAT parent without publishing what it cost is asking the reader to take the trade on faith. Harness, conditions and retention figures will be stated here — including any benchmark where retention is poor.

⚠️ Sampling — read before filing a bug

Do not use greedy decoding. At temperature 0 / top_k 1 the model fails to terminate: it emits its answer, re-opens a response segment and restates it on an exact 40-token cycle, never producing an end-of-generation token.

Follow Moonshot's published guidance:

{ "temperature": 1.0, "top_p": 0.95 }

Measured on this file, five seeds each, identical prompt:

This is greedy-decoding degeneration, not a defect in the quantization. Moonshot's card specifies temperature = 1.0; generation_config.json sets eos_token_id: 163586 (<|end_of_msg|>). Greedy was never a supported operating point for K3.

Deployment notes

Build the PR branch — mainline will not work:

git clone https://github.com/pwilkin/llama.cpp.git && cd llama.cpp
git fetch origin kimi-k3-text && git checkout cf67f0d24511864d2d3da0769108fd6fc16d00d1
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=100 \
      -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j

./build/bin/llama-server \
  -m KIMI-K3-MXP4-DERISKED-Q2_K-00001-of-00038.gguf \
  -ngl 99 -nr -c 8192 --jinja --host 0.0.0.0 --port 8080

• -nr / --no-repack is strongly recommended. Repack is on by default, runs single-threaded (100% of one core, 0% GPU) and allocates a second full copy of the weights.
• Point -m at shard 00001 — llama.cpp resolves the remaining 37 automatically.
• max_tokens. Thinking is always on; the answer lands in content, the chain of thought in reasoning_content. Budget generously or content comes back empty with finish_reason: length.
• Integrity. Verify shard count and byte totals after download before attributing a load failure to the weights.

Benign warning at load

load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect

Ignore it. llama-vocab.cpp emits the warning and inserts the token on the next line — it is self-healing. This file's vocab metadata is correct: [BOS] 163584 · [EOS] 163585 · <|end_of_msg|> 163586 · [EOT] 163593. Structural markers are <|open|> 163587 · <|close|> 163588 · <|sep|> 163589.

File naming

Shards carry the legacy KIMI-K3-MXP4-DERISKED-Q2_K-* prefix, which contains a historic MXP4 typo. Filenames are deliberately not renamed so existing download paths and scripts keep working.

⚠️ Security: structural markers in untrusted input

K3 builds prompts in Python via encoding_k3.py::build_chat_segments, where each segment carries its own allow_special flag (default False). A Jinja template cannot express that, and llama.cpp tokenizes the rendered prompt in a single pass with parse_special=true.

Consequence: a literal <|end_of_msg|> in user content becomes control token 163586 in llama.cpp, where Moonshot's own tokenizer keeps it as ordinary text. Untrusted input can forge chat structure.

This is a general llama.cpp property, not specific to K3 or this quant. If you feed untrusted text to this model, strip or escape these four markers first: <|open|> · <|sep|> · <|close|> · <|end_of_msg|>

Disclaimer

Refusal behaviour in this checkpoint has been deliberately modified at the weight level. It is not a safety-stock model and must not be deployed, marketed, or evaluated as one.

No warranty of any kind. Provided "as is", without warranty express or implied, including fitness for a particular purpose. Nothing here guarantees that any given input will be accepted or refused, that any capability is retained, or that any category of output is unreachable.

Measurements describe what was measured. Throughput, termination and retention figures reflect specific harnesses under specific conditions. They are not safety proofs and do not generalise to multimodal, tool-use, long-context or multi-turn adversarial settings.

Modification by a recipient voids this characterization. Blackfrost's obligations attach at the point of release. Any further ablation, fine-tuning, merging, quantization or alteration by a recipient produces an artifact Blackfrost has not evaluated and does not stand behind — responsibility for that artifact transfers entirely to whoever produced it.

Operator-owned policy. Open weights mean the operator sets and enforces policy. Deploy only in controlled environments with access control, independent logging and review. Do not deploy where refusal behaviour equivalent to upstream Kimi K3 is assumed or required.

Access & licensing

This repository is free and ungated — no request, no review, no purchase. It is published openly for the community to evaluate. The full-precision builds listed above are the paid tier; those repositories are gated and their gates open on a completed licence agreement.

• Base licence: Kimi K3 — Moonshot AI's terms apply to this derivative and travel with it.
• Redistribution: do not redistribute weights outside your grant.
• Evaluation recommendation: should not be evaluated by processes that assume refusal behaviour equivalent to the parent.
• Ask us about other quant points, or calibration against your own threat model.

Contact Blackfrost