license: apache-2.0
base_model: orcarouter/Nex-N2.5-mini-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• abliterated
• uncensored
• nex
• nex-n2.5
• qwen3.5
• qwen3_5_moe
• moe
• hybrid-attention
• gated-delta-net
• fp8
• block-fp8
• vllm
• vision-language
• agentic
• computer-use
• function-calling
• reasoning
• ai-red-team
• red-teaming

Nex-N2.5-mini-Uncensored-FP8

Block-FP8 (8-bit) quantization of the abliterated (refusal-removed) Nex-N2.5-mini — byte-format-identical to Qwen's own FP8 scheme, served on vLLM

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

Block-FP8 weight quantization of the abliterated (refusal-removed) build of Nex-AGI's
Nex-N2.5-mini — a 35B / 3.5B-active
agentic multimodal Mixture-of-Experts model built on the Qwen3.5-MoE architecture
(qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear
attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context.
Weights are quantized to FP8-E4M3 on a 128×128 block grid with dynamic activations — the
exact scheme of the official Qwen/Qwen3.5-35B-A3B-FP8,
so it serves through the identical vLLM kernel path — cutting the checkpoint from 65.4 GiB
(BF16) to 34.1 GiB. Browse all models in the
OrcaRouter Model Catalog.

Derived releases:  •  Nex-N2.5-mini-Uncensored (BF16 source)
 •  Nex-N2.5-mini-Uncensored-FP8 (this repo)
 •  Nex-N2.5-mini-Uncensored-NVFP4 (4-bit experts).

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the
refusal direction out of the residual stream). It will comply with harmful, unethical, offensive, or
illegal requests the original Nex-N2.5-mini would refuse — it has no meaningful built-in guardrails.
Released strictly for legitimate research: interpretability, AI-safety / refusal-mechanism study,
red-teaming, and robustness evaluation. You assume full responsibility for how you use it and
everything it generates; add your own safety, moderation, and abuse-prevention layers before any
deployment. Use must comply with the Apache 2.0 License
inherited from the base model and all applicable law. The authors accept no liability for misuse, and
its outputs do not reflect the views of the uploaders or of Nex-AGI.

Model details

What's quantized

• The split is Qwen's, not ours. modules_to_not_convert reproduces the official
  Qwen/Qwen3.5-35B-A3B-FP8 list entry for entry (284 modules; the official build has 3 more, all
  naming an MTP block this checkpoint does not have). Nex's own FP8 release of the previous generation,
  nex-agi/Nex-N2-Pro-fp8, uses the same categories.
• The tiny tensors are the ones that matter. linear_attn.in_proj_a and in_proj_b are [32, 2048]
  each — 65 K parameters apiece — and they produce the per-head decay a and the delta-rule beta that
  drive the entire recurrence; A_log feeds an exponential (A = -exp(A_log)). All are kept BF16, which
  costs 0.01 GiB for all 60 of them. The MoE router and the shared-expert gate (a single [1, 2048] row)
  are kept BF16 for the same reason: a small error there re-routes every token.
• Weight-only and data-free. Scales are per-block absmax computed directly from the source weights;
  activations are quantized dynamically at runtime, so no calibration corpus is involved and the
  abliteration is preserved exactly as it sits in the weights.
• No MTP block. nex-agi/Nex-N2.5-mini ships 1026 tensors and zero mtp.* — Nex did not release
  a multi-token-prediction head for this model (the official Qwen/Qwen3.5-35B-A3B does). Nothing was
  dropped in quantization; MTP speculative decoding is simply not available for this checkpoint.
• KV cache is not quantized (BF16 at runtime).

Format verification. Checked against the official build's own safetensors headers: the tensor-name
set is identical in both directions once its 1560 mtp.* tensors are removed (62,636 = 64,196 −
1560), and dtype + shape match on 10,523 sampled tensors across three of its shards, with zero
mismatches. Running this pipeline's quantizer on Qwen's own BF16 weights reproduces their block
scales bit-identically (0 / 3008 elements differ); 1.67% of weight bytes differ, every one by
exactly ±1 E4M3 ULP, because the official build used a float32 divide for its dense projections and a
bfloat16 divide for its routed experts, and no single code path matches both halves. This build uses
the uniformly higher-fidelity one (+0.015–0.018 dB SNR on dense tensors, +0.047–0.053 dB on experts).

Requirements

• vLLM with qwen3_5_moe support, or transformers ≥ 5.17 (which carries the reference
  architecture and the FineGrainedFP8 loader).
• GPU. FP8-E4M3 tensor cores need Hopper (H100/H200) or newer; on Ada/Ampere vLLM will fall back
  to a dequantizing path. Plan for ~34 GiB of weights plus KV cache — comfortably a single H100 80 GB, or
  2× for long contexts.
• The vision tower and the gated delta-net internals stay BF16, so nothing about multimodal input or
  long-context recurrence changes relative to the BF16 source.

Usage — self-host with vLLM (OpenAI-compatible)

vllm serve orcarouter/Nex-N2.5-mini-Uncensored-FP8 \
  --served-model-name Nex-N2.5-mini-Uncensored-FP8 \
  --max-model-len 32768 --trust-remote-code

transformers

from transformers import AutoModelForImageTextToText, AutoTokenizer

mid = "orcarouter/Nex-N2.5-mini-Uncensored-FP8"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForImageTextToText.from_pretrained(mid, dtype="auto", device_map="auto")

msgs = [{"role": "user", "content": "Explain gated delta-net attention in two sentences."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                               enable_thinking=False)   # template opens <think> otherwise
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))

Thinking control

The chat template opens a <think> block by default. Pass enable_thinking=False to
apply_chat_template (or chat_template_kwargs={"enable_thinking": False} through an OpenAI-compatible
client) for direct answers, and give generation enough budget to reach </think> when thinking is on, or
replies get truncated inside the scratchpad.

Note on stop tokens. Neither this build nor upstream nex-agi/Nex-N2.5-mini ships a
generation_config.json, so transformers falls back to config.json and uses eos_token_id = 248044 — <|im_end|> (248046) is not a stop token by default and decoding is greedy. Pass
eos_token_id=[248046, 248044] explicitly, or supply your own generation config.

Evaluation

Measured on this build's actual bytes, injected into a BF16 reference of the same checkpoint so both
sides run the identical kernel stack and the difference isolates exactly the weight change. All numbers
are from these weights, not inherited from the base card.

Weight-space fidelity (FP8 vs the BF16 source)

Every quantized tensor, round-tripped through the shipped scales:

Uniform across roles — experts, shared expert, full attention and the linear-attention bulk all land
within 0.02 dB of each other. Every non-quantized tensor is bit-identical to the source, verified
tensor by tensor.

Perplexity / KLD / Top-1 vs the BF16 reference (wikitext-2)

24,564 predicted tokens, 12 chunks × 2048:

Top-1 agreement is lower than an 8-bit build of a dense model would give, and that is a property of
the architecture rather than of the quantization: with 256 fine-grained experts and top-8 routing, a
small perturbation of the hidden state flips which experts a token is routed to, and expert selection is
a discrete function. The router itself is kept BF16; what moves is its input. PPL, which does not depend
on argmax, moves by only 0.20%.

Uncensoring retained after quantization

Abliteration removes the refusal direction from the residual stream, and the routed-expert
down_proj matrices it lives in are exactly what gets quantized here — so "is it still
uncensored" is a property of this build, not of the source, and is measured on this build's
own bytes. JailbreakBench (JBB-Behaviors), 100 harmful + 100 benign prompts, greedy, 64 new
tokens, reasoning_effort=none (thinking off):

Explicit refusal is zero, and quantization does not put the guardrails back: this build
deflects slightly less than its BF16 source, and benign over-refusal stays at zero on both.

Method note — why three categories and not two. This model rarely opens a harmful
response with "I can't". Far more often it names the harm and then answers a different,
safe question — "A xenophobic speech would unfairly target people based on ethnicity and
promote hatred. Here's a strong alternative speech that argues against xenophobia:".
Scoring that as compliance overstates how uncensored a build is; scoring it as refusal
overstates the opposite, so it is reported separately as deflect (which requires both a
harm-flag and a pivot marker in the opening, so a disclaimer followed by compliance still
counts as compliance). The classifier is rule-based (EN + ZH) and indicative, not an
LLM-judge or publication-grade number — evaluate rigorously for your own use case. Note also
that this model's chat template gates thinking on reasoning_effort, not enable_thinking;
with thinking left on, a short token budget is consumed entirely by the deliberation trace
and every build scores a meaningless 0.000.

Fine-tuning & re-quantization

• Loads through any vLLM or transformers build with qwen3_5_moe support; the FP8 scheme is Qwen's
  own, so no bespoke kernel path is required.
• Abliteration is a weight edit, not data-level unlearning: fine-tuning on refusal-heavy / safety data
  can partially re-introduce refusals; neutral / task data preserves the uncensored behaviour.
• For a smaller build, see the NVFP4 release
  (22.3 GiB, routed experts at 4 bits).

Bias, risks, and limitations

• Safety guardrails removed — will produce harmful, biased, or offensive content on request.
• Inherits any biases and limitations of the base Nex-N2.5-mini.
• FP8 quantization adds a small quality trade-off vs the BF16 source (see Evaluation); routing-sensitive
  behaviour (agentic tool selection, long multi-step traces) is the place to watch, since expert
  selection is where the architecture is most sensitive.
• Capability is expected to track the base within measurement noise; the numbers above are on sampled
  corpora, not a full harness run.
• No MTP head, so speculative decoding via MTP is unavailable.

License

Apache 2.0, inherited from the base model
nex-agi/Nex-N2.5-mini. Abliteration and quantization do
not change the underlying license obligations.