base_model:
- huihui-ai/Huihui-Nex-N2-mini-abliterated
- nex-agi/Nex-N2-mini
base_model_relation: quantized
language: - en
- ja
tags: - qwen3_5_moe
- nvfp4
- w4a4
- compressed-tensors
- llm-compressor
- abliterated
- uncensored
- vllm
- blackwell
- sm120
- speculative-decoding
- mtp
- text-only
library_name: vllm
pipeline_tag: text-generation
license: apache-2.0
quantized_by: Lna-Lab
Huihui-Nex-N2-mini-abliterated-text-MTP-NVFP4
Text-only NVFP4 (W4A4) quantization of huihui-ai/Huihui-Nex-N2-mini-abliterated — the abliterated Nex-N2-mini (35B-A3B qwen3_5_moe hybrid-linear MoE). This is the language model only (Qwen3_5MoeForCausalLM, vision tower dropped) at ~22.7 GB, with the native MTP draft in MTP/. For the full vision-language version, see Huihui-Nex-N2-mini-abliterated-MTP-NVFP4.
Made by quantizing the huihui-ai abliterated release with llm-compressor + compressed-tensors. All credit for the model itself goes to huihui-ai (abliteration) and nex-agi (the original Nex-N2-mini); this repo only adds the FP4 quantization.
Lineage: nex-agi/Nex-N2-mini → huihui-ai abliteration → Lna-Lab NVFP4 W4A4, text-only (this repo).
What it is
| Architecture | Qwen3_5MoeForCausalLM (qwen3_5_moe_text) — language model only, no vision tower |
| Total / active | ~34B total · ~3B active (256 experts, top-8, + shared expert) |
| Attention | hybrid: linear-attention (GatedDeltaNet-style SSM) ×3 → full-attention every 4th layer (40 layers) |
| MTP | native draft shipped in MTP/ (BF16, ~1.69 GB) |
| Quantization | NVFP4 nvfp4-pack-quantized, W4A4, group size 16 — 30,720 expert projections + attention/linear-attn projections packed; router/norms/conv/lm_head BF16 |
| Size | ~22.7 GB |
| Nature | abliterated / uncensored · reasoning model (<think>…</think>) |
⚠️ Serving note (read this)
As of vLLM 0.22.0, the standalone text config (model_type: qwen3_5_moe_text / Qwen3_5MoeForCausalLM) is not yet served directly — vLLM routes this architecture through its Qwen3-VL path and expects a multimodal config with vision_config, so it stops with a config-type mismatch. Until vLLM adds a standalone text path for qwen3_5_moe, run the vision-language sibling with --limit-mm-per-prompt '{"image":0,"video":0}' for identical text-only behavior (the language weights are byte-for-byte the same). This text-only repo is published for transformers use, future/other backends, and as the smaller checkpoint.
llm-compressor recipe: QuantizationModifier(targets="Linear", scheme="NVFP4", ignore=["lm_head","re:.*visual.*","re:.*mlp.gate$","re:.*mlp.shared_expert_gate$"]), 32 calibration samples (neuralmagic/calibration).
License
Inherits apache-2.0 from the upstream models. Abliterated/uncensored: you are responsible for how you use it.
Credits
- Original model: nex-agi/Nex-N2-mini
- Abliteration (uncensoring): huihui-ai/Huihui-Nex-N2-mini-abliterated
- NVFP4 quantization: Lna-Lab · Tooling: llm-compressor / compressed-tensors / vLLM
Support the Base Model Author (huihui-ai)
If you find the abliterated base useful, please support huihui-ai — this repo only adds the FP4 quantization; the abliteration work is theirs:
- Ko-fi: https://ko-fi.com/huihuiai
- Bitcoin:
bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge