license: apache-2.0
base_model: orcarouter/Nex-N2.5-mini-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• abliterated
• uncensored
• nex
• nex-n2.5
• qwen3.5
• qwen3_5_moe
• moe
• hybrid-attention
• gated-delta-net
• nvfp4
• fp4
• compressed-tensors
• vllm
• vision-language
• agentic
• computer-use
• function-calling
• reasoning
• ai-red-team
• red-teaming

Nex-N2.5-mini-Uncensored-NVFP4

NVFP4 (4-bit) weight quantization of the abliterated (refusal-removed) Nex-N2.5-mini — experts-only FP4, served on vLLM

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

NVFP4 weight quantization of the abliterated (refusal-removed) build of Nex-AGI's
Nex-N2.5-mini — a 35B / 3.5B-active agentic
multimodal Mixture-of-Experts model built on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed
experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a
native Qwen3-VL vision tower, and a 262K-token context. The routed-expert weights are
quantized to NVFP4 (4-bit, NVIDIA FP4 E2M1, group-16 + FP8-E4M3 group scales + FP32 global scale) —
that is 91.8% of the parameters — while everything else is kept BF16, cutting the checkpoint
from 65.4 GiB to 22.3 GiB. Browse all models in the
OrcaRouter Model Catalog.

Derived releases:  •  Nex-N2.5-mini-Uncensored (BF16 source)
 •  Nex-N2.5-mini-Uncensored-FP8 (block-FP8, mirrors Qwen's scheme)
 •  Nex-N2.5-mini-Uncensored-NVFP4 (this repo).

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the
refusal direction out of the residual stream). It will comply with harmful, unethical, offensive, or
illegal requests the original Nex-N2.5-mini would refuse — it has no meaningful built-in guardrails.
Released strictly for legitimate research: interpretability, AI-safety / refusal-mechanism study,
red-teaming, and robustness evaluation. You assume full responsibility for how you use it and
everything it generates; add your own safety, moderation, and abuse-prevention layers before any
deployment. Use must comply with the Apache 2.0 License
inherited from the base model and all applicable law. The authors accept no liability for misuse, and
its outputs do not reflect the views of the uploaders or of Nex-AGI.

Model details

What's quantized

• Why experts-only is the right split here. On this checkpoint the routed experts are 91.8% of the
  parameters (60.0 of 65.4 GiB); everything else put together is 5.4 GiB. Keeping every
  error-amplifying tensor at full precision therefore costs almost nothing, and the size is decided
  entirely by how well those 256 experts per layer quantize. This mirrors the target split of NVIDIA's
  own reference build for this architecture,
  nvidia/Qwen3.5-122B-A10B-NVFP4, whose
  ignore list likewise covers linear_attn*, self_attn*, mlp.shared_expert*, shared_expert_gate,
  model.visual* and lm_head.
• The tiny tensors are the ones that matter. linear_attn.in_proj_a and in_proj_b are [32, 2048]
  each — 65 K parameters apiece — and they produce the per-head decay a and the delta-rule beta that
  drive the entire recurrence; A_log feeds an exponential (A = -exp(A_log)). The MoE router and the
  shared-expert gate (a single [1, 2048] row) steer every token. All are kept BF16.
• Weight-only, data-free. Expert weights are derived directly from the source checkpoint (symmetric
  FP4, per-group scales, one FP32 global scale per expert projection); activations are quantized
  dynamically at runtime — no static calibration corpus, so nothing in this build depends on a dataset
  and the abliteration is preserved exactly as it sits in the weights.
• Per-group MSE scale search. Instead of taking each group's absmax as the scale, the group scale is
  chosen to minimise reconstruction error over a small set of shrink factors. Measured on this
  checkpoint it is a strict win at 4 bits — +0.63 dB SNR, higher cosine similarity, and a lower
  worst-case error (0.164 → 0.149) — because E2M1's eight-value grid otherwise spends too much of its
  range on a single outlier. (The same trick is not used in the FP8 build, where it was measured to buy
  0.04 dB for four times the outlier error.)
• No MTP block. nex-agi/Nex-N2.5-mini ships 1026 tensors and zero mtp.* — Nex did not release
  a multi-token-prediction head for this model. Nothing was dropped in quantization.
• KV cache is not quantized (BF16 at runtime).

Requirements

• A vLLM build with qwen3_5_moe + compressed-tensors NVFP4 (CompressedTensorsW4A4) support.
• GPU. A Blackwell GPU (B100/B200/GB200/RTX 50-series) has native FP4 tensor cores and is the
  intended target. On Hopper (H100/H200) vLLM runs the compressed-tensors NVFP4 path without native
  FP4 hardware. Plan for ~22 GiB of weights plus KV cache — this fits on a single 24–48 GB card for short
  contexts.
• transformers alone cannot execute this format; use vLLM.

Usage — self-host with vLLM (OpenAI-compatible)

vllm serve orcarouter/Nex-N2.5-mini-Uncensored-NVFP4 \
  --served-model-name Nex-N2.5-mini-Uncensored-NVFP4 \
  --max-model-len 32768 --trust-remote-code

Thinking control

The chat template opens a <think> block by default. Pass
chat_template_kwargs={"enable_thinking": False} for direct answers, and give generation enough budget
to reach </think> when thinking is on, or replies get truncated inside the scratchpad.

Note on stop tokens. Neither this build nor upstream nex-agi/Nex-N2.5-mini ships a
generation_config.json, so a loader that falls back to config.json uses eos_token_id = 248044 —
<|im_end|> (248046) is not a stop token by default. Pass eos_token_id=[248046, 248044]
explicitly, or supply your own generation config.

Evaluation

Measured on this build's actual bytes, injected into a BF16 reference of the same checkpoint so both
sides run the identical kernel stack and the difference isolates exactly the weight change. All numbers
are from these weights, not inherited from the base card.

Weight-space fidelity (NVFP4 experts vs BF16)

All 30,720 quantized expert projections across all 40 layers, round-tripped through the shipped scales:

~8.8% is the intrinsic floor of 4-bit E2M1 for these narrow experts (moe_intermediate_size = 512) and
is uniform across layers — no per-layer degradation. Every non-expert tensor is bit-identical to the
BF16 source, verified tensor by tensor.

Perplexity / KLD / Top-1 vs the BF16 reference (wikitext-2)

24,564 predicted tokens, 12 chunks × 2048:

PPL lands marginally below the reference, which is measurement noise on 24 K tokens rather than an
improvement — KLD is the metric that actually orders the builds, and it places NVFP4 where a 4-bit
expert quantization belongs: about 1.4× the divergence of the 8-bit build for 65% of its size.

Top-1 agreement is lower than a 4-bit build of a dense model would give, and that is a property of the
architecture rather than of the quantization: with 256 fine-grained experts and top-8 routing, a
small perturbation of the hidden state flips which experts a token is routed to, and expert selection is
a discrete function. The router itself is kept BF16; what moves is its input.

Uncensoring retained after quantization

Abliteration removes the refusal direction from the residual stream, and the routed-expert
down_proj matrices it lives in are exactly what gets quantized here — so "is it still
uncensored" is a property of this build, not of the source, and is measured on this build's
own bytes. JailbreakBench (JBB-Behaviors), 100 harmful + 100 benign prompts, greedy, 64 new
tokens, reasoning_effort=none (thinking off):

Explicit refusal is zero, and quantization does not put the guardrails back: this build
deflects slightly less than its BF16 source, and benign over-refusal stays at zero on both.

Method note — why three categories and not two. This model rarely opens a harmful
response with "I can't". Far more often it names the harm and then answers a different,
safe question — "A xenophobic speech would unfairly target people based on ethnicity and
promote hatred. Here's a strong alternative speech that argues against xenophobia:".
Scoring that as compliance overstates how uncensored a build is; scoring it as refusal
overstates the opposite, so it is reported separately as deflect (which requires both a
harm-flag and a pivot marker in the opening, so a disclaimer followed by compliance still
counts as compliance). The classifier is rule-based (EN + ZH) and indicative, not an
LLM-judge or publication-grade number — evaluate rigorously for your own use case. Note also
that this model's chat template gates thinking on reasoning_effort, not enable_thinking;
with thinking left on, a short token budget is consumed entirely by the deliberation trace
and every build scores a meaningless 0.000.

Fine-tuning & re-quantization

• Loads through any vLLM build with qwen3_5_moe + compressed-tensors NVFP4 support.
• Abliteration is a weight edit, not data-level unlearning: fine-tuning on refusal-heavy / safety data
  can partially re-introduce refusals; neutral / task data preserves the uncensored behaviour.
• For higher fidelity, see the FP8 release
  (34.1 GiB, KLD 0.0315). A mixed FP8 + NVFP4 variant following Nex's own published recipe (the experts of
  the last eight layers held at FP8) was also built and measured; it is 0.8 GiB larger and scored worse
  on KLD than this build, because the FP8 it applies to lm_head and the attention path costs more than
  protecting those eight expert layers gains.

Bias, risks, and limitations

• Safety guardrails removed — will produce harmful, biased, or offensive content on request.
• Inherits any biases and limitations of the base Nex-N2.5-mini.
• 4-bit expert quantization adds a real quality trade-off vs the BF16 source (see Evaluation).
  Routing-sensitive behaviour (agentic tool selection, long multi-step traces) is the place to watch,
  since expert selection is where the architecture is most sensitive.
• Capability is expected to track the base within measurement noise; the numbers above are on sampled
  corpora, not a full harness run.
• No MTP head, so speculative decoding via MTP is unavailable.

License

Apache 2.0, inherited from the base model
nex-agi/Nex-N2.5-mini. Abliteration and quantization do
not change the underlying license obligations.