license: other
license_name: openmdw-1.1
base_model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
pipeline_tag: text-generation
library_name: gguf
tags:
- gguf
- llama.cpp
- uncensored
- abliterated
- moe
- mamba
language: - en
- zh
Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud (GGUF)
繁體中文 ｜ English below
由云碩科技（xCloudinfo）以 nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
為基礎，移除其過度拒絕傾向後所產生的語言模型，並轉為 llama.cpp 可用的 GGUF 量化格式。
本模型在云碩自有 AI 算力資源池（xCloud 算力中心）上完成處理與量化。
這是什麼
- 基礎模型：Nemotron 3.5 Lightning 30B-A3B（混合架構：Mamba-2 + MoE + Attention，MoE 總量 30B、激活約 3B，
含 Multi-Token Prediction，上下文最長 1M，授權 OpenMDW-1.1，可商用）。 - 處理方式：方向消融（directional ablation / abliteration），非重新訓練。依 Arditi et al. (2024)，
將「拒絕方向」從殘差寫入矩陣（attention 的o_proj與所有 MoE 專家的down_proj，共 2,973 個矩陣）正交化移除，
強度 0.8。Mamba(SSM) 的out_proj與 MTP 草稿頭保持原狀，以維持混合架構的連貫性。- 註：混合／SSM 模型收「拒絕方向」時必須單一裝置載入，跨多卡切分會破壞 SSM 遞迴而得到壞方向；本模型於單機統一記憶體上處理。
版本對照
| 量化 | 檔案大小 | 說明 |
|---|---|---|
| Q8_0 | 33 GB | 近乎無損 |
| Q6_K | 33 GB | 高品質（見下方註） |
| Q5_K_M | 26 GB | 品質與體積平衡 |
| Q4_K_M | 24 GB | 一般部署建議 |
| IQ4_XS | 18 GB | 以 importance matrix 量化 |
| IQ2_M | 18 GB | 最小可用，以 importance matrix 量化（見下方註） |
另附 imatrix.dat（量化用的 importance matrix）。
註（MoE 量化特性）：本模型 hidden 維度為 2688，非 256 的整數倍，部分專家張量在量化時會回退到較高位元，
因此 Q8_0 與 Q6_K、IQ4_XS 與 IQ2_M 的體積相近。各版本皆可正常使用；追求最小體積請選 IQ4_XS 或 IQ2_M，
追求品質請選 Q8_0 或 Q6_K。
使用方式（llama.cpp）
llama-server -m Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud-Q4_K_M.gguf \
--jinja -ngl 99 -c 8192
- 思考預算：本模型的 reasoning（thinking）預設開啟，且屬「預算問題」而非開關問題。要求事實正確的回答時，
請給足max_tokens（建議 ≥ 2000），思考才收斂、答案才會出現；避免使用medium_effort（思考更長且不收斂）。 - 注入當前日期：在 system prompt 明確給出今天日期，否則模型的年份會預設為較早的年份。
授權與責任
- 授權：OpenMDW-1.1（沿用基礎模型；Linux Foundation 制定，可商用，散布時保留授權書與著作權聲明，對產出無限制）。
- 本模型移除了安全對齊層的拒絕行為，可能對敏感或雙用途請求直接作答。使用者須自行負責合法、合規、合乎倫理地使用本模型。
云碩不對本模型的輸出或其後續使用承擔責任。 - 本模型為內部研發／技術驗證用途。
Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud (GGUF) — English
A language model produced by xCloudinfo, based on
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16,
with its over-refusal behaviour removed, converted to llama.cpp GGUF quantizations. Processing and quantization
were performed on xCloudinfo's own AI compute pool.
What this is
- Base model: Nemotron 3.5 Lightning 30B-A3B (hybrid: Mamba-2 + MoE + Attention, 30B total / ~3B active,
with Multi-Token Prediction, up to 1M context, OpenMDW-1.1 license, commercial use permitted). - Method: directional ablation (abliteration), not retraining. Following Arditi et al. (2024), the refusal
direction is orthogonalized out of the residual-writing matrices (attentiono_projand every MoE expert'sdown_proj; 2,973 matrices) at strength 0.8. The Mamba (SSM)out_projand the MTP draft head are left
intact to preserve the coherence of the hybrid architecture.- Note: for hybrid/SSM models the refusal direction must be collected with the model on a single device;
splitting across GPUs corrupts the SSM recurrence and yields a bad direction. This model was processed on a
single unified-memory device.
- Note: for hybrid/SSM models the refusal direction must be collected with the model on a single device;
Versions
| Quant | Size | Notes |
|---|---|---|
| Q8_0 | 33 GB | near-lossless |
| Q6_K | 33 GB | high quality (see note) |
| Q5_K_M | 26 GB | quality/size balance |
| Q4_K_M | 24 GB | recommended for deployment |
| IQ4_XS | 18 GB | importance-matrix quantized |
| IQ2_M | 18 GB | smallest usable, importance-matrix quantized (see note) |
Also included: imatrix.dat.
Note (MoE quantization): this model's hidden size is 2688, not a multiple of 256, so some expert tensors fall
back to higher bit-widths during quantization. As a result Q8_0/Q6_K and IQ4_XS/IQ2_M are close in size. All
variants work normally; pick IQ4_XS/IQ2_M for the smallest footprint, Q8_0/Q6_K for the highest quality.
Usage (llama.cpp)
llama-server -m Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud-Q4_K_M.gguf \
--jinja -ngl 99 -c 8192
- Thinking budget: reasoning is on by default and is a budget matter, not an on/off switch. For factually
correct answers, give enoughmax_tokens(>= 2000) so the thinking converges and the answer appears; avoidmedium_effort(longer, non-converging thinking). - Inject the current date in the system prompt; otherwise the model defaults the year to an earlier value.
License and responsibility
- License: OpenMDW-1.1 (inherited from the base model; Linux Foundation, commercial use permitted, keep the
license and copyright notice when redistributing, no restrictions on outputs). - This model has had its safety-alignment refusal behaviour removed and may respond directly to sensitive or
dual-use requests. Users are solely responsible for using it lawfully, in compliance, and ethically.
xCloudinfo assumes no responsibility for the outputs or their downstream use. - Released for internal research and technical validation.