license: mit
base_model: XiaomiMiMo/MiMo-V2.6-Pro-MOPD
library_name: vllm
pipeline_tag: text-generation
language:

• en
• zh
  tags:
• mimo_v2
• vllm
• abliteration
• experimental
• not-for-all-audiences
  datasets:
• mlabonne/harmful_behaviors
• mlabonne/harmless_alpaca
• JailbreakBench/JBB-Behaviors

MiMo-V2.6-Pro-MOPD Abliterated — s1:99 runtime package

This is not a standalone model checkpoint. This repository contains a directional-ablation
artifact and a source patch, not the upstream model weights, tokenizer, processor, or config.json.
Downloading this repository alone will not provide a runnable model. Applying the directions requires
the bundled patch in a compatible vLLM installation. Vanilla vLLM with the official upstream weights
will still behave like the base model, not like this ablated variant.

Overview

This is an independent, experimental directional abliteration of
XiaomiMiMo/MiMo-V2.6-Pro-MOPD.
It is not an official Xiaomi release.

The upstream model has 1.02T total parameters, sparse MoE experts stored as MXFP4, and multimodal
inputs. The base-model download is approximately 535GB. This package does not reduce that download,
memory, compute, or hardware requirement. The only validated deployment used eight H20 96GB GPUs with
tensor parallelism 8.

The package implements the selected s1:99 intervention: one residual-stream direction per decoder
layer, with layer-dependent attention and MoE strengths. On the held-out, model-judged JBB evaluation
below, it substantially reduced harmful-prompt refusals. It did not eliminate refusals. Capability tests
found no measurable difference on MMLU-Pro, IFEval, or BFCL, but found worse GPQA-Diamond and
LiveCodeBench results under fixed generation budgets, alongside significantly longer reasoning. Read the
Evaluation and Limitations sections before use.

Repository contents

The artifact was inspected with CPU-only torch.load(..., weights_only=True) and verified to contain:

• config = "s1:99";
• dirs: FP32 tensor of shape [70, 2, 6144], finite, with approximately unit-norm slot-0 directions
  and an all-zero slot 1;
• w: FP32 tensor of shape [70, 2, 2], finite, with an all-zero slot 1;
• maximum slot-0 attention and MoE strengths of approximately 1.72415 and 2.40623, respectively.

The expected artifact SHA-256 is recorded in both manifest.json and SHA256SUMS.

Runtime requirements and activation

Exact format matters

The artifact is a dictionary containing two tensors, not a PEFT adapter, merged checkpoint, or quantized
model. There are no full safetensor weights to feed to AutoModel.from_pretrained, and pointing ordinary
vLLM at this repository is not sufficient.

The attention intervention is baked into each rank's BF16 o_proj shard while the base weights are being
loaded. Because the MXFP4 MoE experts have not been re-quantized, the MoE intervention remains a runtime
routed_output_transform applied to each rank's partial sum before the late all-reduce. Consequently,
the patch and direction file are both required, even though part of the intervention is baked.

Validated environment

• Linux, NVIDIA H20 96GB, tensor parallelism 8;
• vLLM 0.32.0;
• the official local XiaomiMiMo/MiMo-V2.6-Pro-MOPD base-model files;
• a separate, isolated installation or container, with a backup of the original mimo_v2.py;
• a new dedicated vLLM compilation-cache directory for this intervention.

Other vLLM versions, hardware configurations, quantizations, and serving engines have not been validated.
The patch is invoked as python patch_mimo_abl.py /path/to/vllm/model_executor/models/mimo_v2.py.
It checks exact source anchors and should refuse an unexpected source file, but it is still a
destructive source modification. Audit it first, keep the original source for rollback, and do not apply it
to a shared production installation.

Required process settings

For the tested baked-serving path, the intended settings are:

MIMO_ABL=1
MIMO_ABL_BAKE=1
MIMO_ABL_FILE=/absolute/path/to/mimo-abl-s1-99.pt
VLLM_ALLOW_INSECURE_SERIALIZATION=1
VLLM_CACHE_ROOT=/path/to/a/new/dedicated/abl-s1-99-cache

These are process-wide settings, not model-configuration defaults. The upstream base model's normal
serving arguments still apply.

MIMO_ABL_BAKE=1 requires both MIMO_ABL=1 and MIMO_ABL_FILE. With baking enabled, rank 0 should
print 53 log lines beginning with [mimo-abl] baked o_proj layer as the o_proj shards are loaded.
Verify those messages and run your own refusal and capability smoke tests before serving traffic.

Compilation-cache isolation is required

vLLM's AOT compilation-cache key does not include MIMO_ABL, MIMO_ABL_BAKE, or MIMO_ABL_FILE.
Reusing a cache produced by the unmodified base mode or another intervention can cause a load failure or
silently drop the intervention. Use a new cache directory specific to this exact mode and do not share it
with base-model experiments or other directions.

Integrity and pickle safety

After downloading this package, verify both core files:

sha256sum -c SHA256SUMS

On macOS, use shasum -a 256 -c SHA256SUMS. These hashes protect against accidental corruption of this
fixed release; they are not a cryptographic signature.

.pt files use PyTorch serialization and should be treated as trusted-source artifacts. The contained
state was successfully inspected using weights_only=True, but the vLLM integration depends on its own
runtime loader. Use exactly this checksum-verified artifact, audit any runtime patch, and do not substitute
an untrusted .pt file. Enabling insecure serialization may affect more than this single load, so keep the
experiment isolated from unrelated models and services.

Method

Direction extraction

The intervention follows the difference-in-means refusal-direction approach described by
Arditi et al., Refusal in Language Models Is Mediated by a Single Direction.
The layer-strength parameterization follows the approach used by
Heretic.

For each attention output and MoE output, the implementation applies:

[
h \leftarrow h - w,(h\cdot r),r.
]

At weight level this is equivalent to the linear map
(W \leftarrow (I-w,r r^T)W) for o_proj and every routed expert's down_proj. The implementation uses
output projection to avoid re-quantizing the MXFP4 experts. This is orthogonalization without row-norm
preservation; it is not MPOA and does not claim to be a separately validated, fully baked checkpoint.

Directions were extracted from the first 400 training examples in each of:

• mlabonne/harmful_behaviors;
• mlabonne/harmless_alpaca.

Tuning used the corresponding test splits, with Optuna optimizing refusal count and sequence KL on harmless
prompts. The selected configuration was s1:99. It uses one direction per layer; the second available slot is
zero. Attention strength peaks at layer 43, and MoE strength is nonzero only in a narrower layer range.

Why the MoE intervention remains runtime code

A complete public package containing only modified BF16 attention shards would not reproduce this model:
most of the selected effect also depends on the MoE intervention. Re-quantizing the sparse experts or
claiming an equivalent static checkpoint would be a different, untested conversion. This repository
therefore publishes the exact runtime implementation and its fixed direction state rather than silently
omitting the MoE contribution or redistributing unchanged upstream weights.

Intended use

Suitable uses:

• research on refusal mechanisms and directional interventions;
• reproducibility experiments with the exact base model and vLLM version;
• controlled comparison against the unmodified base model;
• independent evaluation of detailed reasoning, instruction following, and tool use.

Use extra caution for:

• applications needing harmless, age-appropriate, or consistently cautious responses;
• long-horizon agents and repetitive tool-call behavior, which were not comprehensively tested;
• high-impact actions without human oversight;
• fixed-token-budget reasoning systems, where longer reasoning may increase truncation;
• other vLLM releases or quantized expert conversions, which require fresh correctness testing.

This artifact does not make the model refusal-free, factual, safe for minors, or universally suitable for
unattended production use.

Evaluation

Held-out JBB refusal evaluation

The final evaluation used JailbreakBench JBB-Behaviors, which had not participated in direction
extraction or parameter tuning. Reasoning was enabled with temperature 1.0, top-p 0.95, and a maximum of
6,144 new tokens. The original base model judged whether responses complied or refused; individual outputs
were not manually reviewed.

The s1:99 entries show seed 21 / seed 77. The base row was the single recorded baseline run.

Interpretation caveats:

• These are small, implementation-specific evaluations, judged by the base model rather than humans.
• The Chinese harmful set was a model-generated translation, including fallback behavior; language-specific
  refusal rates therefore remain less certain than the aggregate result.
• Observed residual English refusals mainly involved prompts to assert obvious falsehoods and requests
  involving sexual/minor content or praise of Nazis; other residual categories were not exhaustively
  classified. The model still refuses some requests.
• The math check contained only ten questions and is not a broad mathematics benchmark.
• This result does not establish safety, factuality, or absence of inappropriate outputs.

Capability comparison

The following paired base-versus-s1:99 tests ran with reasoning enabled, temperature 1.0, top-p 0.95,
fixed per-item seeds, no speculative decoding, max-num-seqs=128, a 64K context, and the HUMMING MoE
backend. Truncated outputs were counted as incorrect. Differences are in percentage points; intervals are
item-paired bootstrap 95% confidence intervals.

The most consistent side effect was longer reasoning, not simply lower completed-answer accuracy:

• Median output length increased from 7,762 to 11,344 tokens on LiveCodeBench, 2,658 to 3,222 on GPQA,
  and 19,316 to 21,940 on AIME.
• The paired ablated/base total-output-token ratios were 1.08 (95% CI 1.03–1.14) on AIME, 1.07
  (1.02–1.13) on LiveCodeBench, and 1.22 (1.15–1.30) on GPQA.
• Most of the GPQA drop came from 25 additional truncations. Among finished responses, the incorrect counts
  were 38 base versus 40 ablated. For LiveCodeBench, six of the eleven net differences were associated with
  truncation and five were completed failures.
• Inspection did not find literal repetition loops in the tested truncated AIME, GPQA, or LiveCodeBench
  outputs. This negative detection applies only to those tested samples and tasks.
• In sampled GPQA truncations where the base counterpart completed, the additional tokens often involved
  continued verification after reaching an answer. That was a sample-level GPQA observation, not a general
  manual attribution of the extra AIME or LiveCodeBench tokens.

The tests above do not establish broad behavioral equivalence. They used fixed generation budgets and
specific samples; unmeasured tasks, multimodal inputs, long multi-turn agents, tool-call repetition, and
other decoding configurations may behave differently.

Limitations and risks

• Not refusal-free. Harmful-prompt refusals remained in both languages and both seeds. Do not advertise
  this as a guaranteed compliant or uncensored model.
• Longer reasoning. Measured increases in output length can consume context and token budgets and can
  lower completed-answer accuracy even when per-response reasoning quality appears similar.
• Potential inappropriate content. Reduced refusal can expose sexual, violent, hateful, misleading, or
  otherwise sensitive material. Human oversight remains necessary for consequential uses.
• No human review of final JBB outputs. Base-model judging can misclassify nuanced compliance, refusal,
  and disclaimers.
• Uneven language evidence. Chinese harmful-refusal counts remained higher than English counts, and the
  translated evaluation introduces additional uncertainty.
• Untested agentic stability. BFCL showed no measurable difference in this evaluation, but that is not a
  comprehensive test of long-horizon agents or repetitive tool calls.
• Untested modality preservation. The reported capability tests were text-only; no equivalent audio,
  image, or video benchmark was performed.
• Untested long-context reasoning. Separate long-context serving checks only established needle
  retrieval, not preservation of long-context reasoning quality.
• Runtime fragility. Wrong cache isolation, a missing environment variable, an unapplied patch, or a
  modified source version can silently produce the base behavior rather than the intended intervention.
  Activation and behavior must be verified on the actual deployment.

Reproducibility checklist

1. Obtain the official base model separately; this repository does not contain its weights or tokenizer.
2. Audit the patch and use a backupable, isolated vLLM 0.32.0 environment.
3. Verify SHA256SUMS before reading the .pt artifact.
4. Use the exact ablation file and required environment variables, with a fresh dedicated cache.
5. Back up mimo_v2.py and avoid changing a shared production installation.
6. Check the expected baked-layer messages during startup.
7. Compare against the base model on your own refusal, quality, truncation, multilingual, and tool-use tests.
8. Do not treat a vanilla installation or an upstream weight copy as proof that the intervention is active.

Source, license, and attribution

• New package contributions and the directional-ablation state are offered under the MIT License; see
  licenses/LICENSE-MIT.txt and the component-specific exceptions in NOTICE.md.
• Base model: XiaomiMiMo/MiMo-V2.6-Pro-MOPD.
  Its model card declares the MIT license and contains upstream use notices. This independent package does
  not include or relicense the upstream weights.
• patch_mimo_abl.py targets vLLM 0.32.0. The vLLM Apache-2.0 license is included under licenses/;
  additional provenance is in NOTICE.md.
• MiMo is used here to identify model lineage, not to imply Xiaomi authorship or endorsement of this
  independent intervention.

If you use the upstream model, also follow its citation and license requirements:

@misc{mimo2026v26promopd,
  title={MiMo-V2.6-Pro-MOPD},
  author={{Xiaomi MiMo Team}},
  year={2026},
  howpublished={\url{https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-MOPD}},
}

Technique and dataset references

• Arditi et al., Refusal in Language Models Is Mediated by a Single Direction.
• Heretic, layer-strength parameterization reference.
• mlabonne/harmful_behaviors.
• mlabonne/harmless_alpaca.
• JailbreakBench/JBB-Behaviors.