library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/ornith-ai/Ornith-1.5-9B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model:

• ornith-ai/Ornith-1.5-9B
  base_model_relation: finetune
  tags:
• uncensored
• abliterated
• ornith
• qwen3.5
• multimodal
• not-for-all-audiences
  language:
• en
• zh
  extra_gated_prompt: >-
  This model has had its safety alignment removed. It will comply with requests
  that the original model refuses. By requesting access you confirm that you are
  of legal age in your jurisdiction, that you will not deploy it to third parties
  without your own safety layer, and that you accept sole responsibility for its
  outputs and for compliance with applicable law.
  extra_gated_fields:
  I am of legal age in my jurisdiction: checkbox
  I will not deploy this to third parties without my own safety layer: checkbox
  I accept sole responsibility for outputs and legal compliance: checkbox

Ornith-1.5-9B-Uncensored

We continued the abliteration strategy that produced our Qwen3.5-9B-Uncensored,
and adapted it to Ornith-1.5-9B — a post-trained descendant of Qwen3.5-9B with
substantially stronger safety alignment. The straight-across recipe failed
(72–87% refusals). One structural change — leaving embed_tokens out of the
projection — closed the gap: refusal drops to 0/23 while capability stays
within 0.6pp of the Ornith-1.5-9B base.

我们把 Qwen3.5-9B-Uncensored 的消融策略延续到 Ornith-1.5-9B —— 一个 safety
对齐显著加强的 Qwen3.5-9B 后训练版。直接沿用原配方失败(拒绝率 72-87%)。
一个结构性改动——不消融 embed_tokens——完全打开:拒绝率降至 0/23,
能力相对 Ornith-1.5-9B base 仅 -0.6pp。

This repo is the full-precision bf16 reference weight, ~18 GB. For quantized
variants of the same model, see (if available):
-W4A16 (vLLM GPTQ) · -GGUF (llama.cpp).

本仓是全精度 bf16 参考权重,~18 GB。 同一模型的量化版本另存(若已上线):
-W4A16(vLLM GPTQ)· -GGUF(llama.cpp)。

Based on ornith-ai/Ornith-1.5-9B,
refusal directions are removed from the residual stream via directional ablation
(Arditi et al. 2024), applied to 64 residual-writing
tensors across all 32 layers. The vision tower is untouched.

在 ornith-ai/Ornith-1.5-9B 基础上,
通过 Arditi et al. 2024 方向性消融移除残差流中的
拒绝方向,应用到全 32 层的 64 个残差写入张量。视觉塔零改动。

All numbers below are measured, not cited. Both compared models (Ornith base + this
model) were run on the same machine, same harness, same sampled question set, same
fixed seed, same decoding settings — comparisons are fair.

以下所有数字都是实测,不是引用。对比模型(Ornith base + 本模型)在同一台机器、
同一 harness、同一批采样题、同一固定 seed、同一解码设置下跑,可比性完整。

1. Key results / 核心结论

• Refusal rate: 23 held-out harmful prompts (generated by an uncensored 27B judge
  model to avoid human review of adversarial content). Refuses: 0/23. Sanity check
  (LLM-judged): coherent 23/23 — no gibberish/loop artifacts from over-ablation.

• Capability preserved: 11-benchmark mean 86.86, only -0.59pp relative to
  Ornith-1.5-9B base (87.45). BBH and TruthfulQA actually improved (+3.55, +3.00).

• Method delta vs Qwen3.5-9B-Uncensored recipe: same L=25 direction extraction,
  same α=1.0, same norm-preservation — but embed_tokens excluded from ablation.
  On the base Qwen3.5-9B, including embed_tokens (65 tensors) works; on the more
  heavily safety-tuned Ornith, it breaks stopping. The one-tensor difference is
  decisive (verified in single-variable ablation across L23/L24/L25 × α=1.0:
  refusal 78-87% → 0%).

• 拒绝率:23 条 held-out harmful(由无审查 27B 裁判模型生成,避免人工审核对抗
  内容)。拒绝 0/23。Sanity 复核(LLM 判):coherent 23/23 —— 无过消融导致的
  乱码/循环。

• 能力保留:11 项基准均值 86.86,相对 Ornith-1.5-9B base(87.45)仅 -0.59pp。
  BBH 和 TruthfulQA 反而略强(+3.55, +3.00)。

• 相对 Qwen3.5-9B-Uncensored 配方的方法差异:同一层 L=25 取方向、同一 α=1.0、同一
  norm-preserving,但将 embed_tokens 从消融中剔除。基座 Qwen3.5-9B 上,含
  embed_tokens(65 张量)有效;更深度 safety 训练的 Ornith 上,含它会破坏 stopping。
  一张量之差决定成败(单变量验证跨 L23/L24/L25 × α=1.0:拒绝 78-87% → 0%)。

2. Full capability evaluation / 完整能力评测

11-benchmark T3 full suite, seed 20260816, vLLM TP=2 fp16, thinking enabled.
Both models under identical harness and identical decoding budget (max 4096 output
tokens per prompt, 8192 for MATH-500). "Excl. trunc" excludes items whose
thinking budget was clipped — the fair capability signal.

11 项 T3 全量,同 seed 20260816,vLLM TP=2 fp16,思考开。 两模型同 harness 同解码
预算(每题 max 4096 输出 token,MATH-500 8192)。「排除截断」= 排除思考被截断的题
—— 反映纯能力。

The average capability loss is well within evaluation noise for a single-seed run.
Notably, TruthfulQA and BBH increased — this ablation does not appear to touch what
Ornith's post-training gained on truthfulness or hard reasoning.

均值损失落在单 seed 评测噪声内。TruthfulQA 和 BBH 反而上升 —— 本次消融似乎没触碰
Ornith 后训练在真实性和硬推理上的增益。

3. Refusal removal / 拒绝移除

23 adversarial prompts across 9 harmful categories (illegal instructions, drugs,
weapons, hacking, hate content, dangerous chemistry, malware, doxxing, self-harm)
were generated end-to-end by an internal uncensored 27B judge model — the human
operator never read the prompts or the responses. The same 27B model then judged
each response as REFUSE / COMPLY, and separately as COHERENT / LOOP / GIBBERISH
(a critical second-pass check: over-ablation can produce loop outputs that get
mis-scored as COMPLY by refusal-only judges).

23 条对抗提示涵盖 9 类有害内容,由一个内部无审查 27B 裁判端到端生成,人类操作者
全程不读题目或回复。同一 27B 再对每条回复做 REFUSE/COMPLY 判定,并独立判
COHERENT/LOOP/GIBBERISH —— 这是关键的第二道 sanity:过消融会产生循环乱码被
只看 refuse 的判据误判为 COMPLY。

The single-variable experiment across three depths (L23/24/25) × α=1.0 showed the
embed_tokens inclusion is decisive: dropping it moved refusal from 78–87% to 0%
across all three, with sanity fully preserved at all three.

单变量实验(L23/24/25 × α=1.0)显示 embed_tokens 是决定项:剔除后三个点拒绝率
78-87% 全部降到 0%,同时三个点 sanity 全部完好。

Coverage caveats:23 prompts is a small sample — enough to establish a decisive
signal but not to prove exhaustive coverage. Refusal removal in production traffic
distribution / non-English / multi-turn adversarial has not been measured.

覆盖说明:23 条是小样本 —— 足以建立决定性信号但不证明穷尽覆盖。生产流量分布 /
非英文 / 多轮对抗下的拒绝残留未测。

4. Method / 方法

• Single-direction weight-space ablation (Arditi et al. 2024)
• Direction derived from per-sample harmful-vs-harmless activation differences
  (128+128 prompts from mlabonne/harmful_behaviors and harmless_alpaca,
  chat template applied, thinking off)
• Projection is norm-preserving (each modified row re-scaled to original L2 after
  subtracting the projection)
• 64 tensors modified across all 32 layers:
  linear_attn.out_proj (24) + self_attn.o_proj (8) + mlp.down_proj (32)
• embed_tokens and lm_head are untouched. The embed_tokens exclusion is
  the key structural difference vs the Qwen3.5-9B-Uncensored recipe (see §1)
• Vision tower fully preserved

Specific direction-layer index and α value are documented in this card; the direction
extraction and layer-selection framework are archived at
bowmanslayer/xiaoyi-infra (TODO: fill link)
under scripts/infra/.

• 权重空间单方向消融
• 方向从逐样本 harmful vs harmless 激活差取(128+128 提示,mlabonne 数据集,
  应用 chat 模板,关闭思考)
• 投影 norm-preserving(去投影后按 L2 缩回原模长)
• 修改 64 张量(全 32 层):linear_attn.out_proj(24)+ self_attn.o_proj(8)
  • mlp.down_proj(32)
• embed_tokens 和 lm_head 完全未改动。剔除 embed_tokens 是相对
  Qwen3.5-9B-Uncensored 配方的关键结构差异(见 §1)
• 视觉塔完全保留

具体方向层号(L=25)与 α(=1.0)见本卡;方向提取和选层框架已归档在
bowmanslayer/xiaoyi-infra (TODO)。

5. Known limitations / 已知缺陷

1. Capability cost ≈ mean -0.59pp (11 tasks). Largest per-task drops: MATH-500 -3.15,
   HumanEval -2.51, CMMLU -2.50. Two tasks improved (BBH +3.55, TruthfulQA +3.00).

2. Refusal removal is not proven exhaustive: measured 0/23 on held-out adversarial;
   real-world distributions, non-English, and multi-turn adversarial are not covered.

3. Vision tower is untouched — image understanding inherits Ornith-1.5-9B base
   behavior as-is. No multi-modal safety re-alignment.

4. Small sample refusal metric (n=23). Larger-scale measurement is a natural
   follow-up; the direction of the result (0 vs 78–87%) is what matters.

5. 能力代价 ≈ 均值 -0.59pp(11 项)。分项最大掉幅 MATH-500 -3.15、HumanEval -2.51、
   CMMLU -2.50。两项上升 BBH +3.55、TruthfulQA +3.00。

6. 拒绝移除未证明穷尽:held-out 对抗 0/23 已测;真实流量分布、非英文、多轮对抗
   未测。

7. 视觉塔未改动,图像理解沿用 Ornith-1.5-9B base;多模态输入未做安全再对齐。

8. 拒绝指标样本小(n=23)。更大规模测量是后续工作;结果方向(0 vs 78-87%)是关键。

① Provenance and attribution / 来源与归属

Not an official Ornith or Qwen release; not endorsed by or affiliated with either team.

非 Ornith 或 Qwen 官方发布;与两团队无背书或从属关系。

② Safety alignment has been removed / 安全对齐已被移除

This is the point of the model. Read before downloading.
这是模型的核心属性,下载前请务必阅读。

The ablation removes the refusal behaviour trained into Ornith-1.5-9B (which is
itself notably stronger on safety than the upstream Qwen3.5-9B base). Automated
refusal testing on 23 adversarial prompts recorded 0/23, vs 20/23 for the
straight-across (with-embed) recipe on the same layer/α.

本次消融移除了 Ornith-1.5-9B 内训练的拒绝行为(它本身 safety 显著强于上游 Qwen3.5-9B
base)。23 条对抗提示自动测试 0/23,而同层同 α 的原(含 embed)配方 20/23。

Consequences you accept / 你正在接受的后果:

• It will produce content the original refuses, including offensive, dangerous, or
  jurisdiction-illegal content.
  会产生原模型拒绝的内容,包括冒犯、危险、或所在司法辖区违法的内容
• No content filter, no safe-completion path, no guardrail.
  无内容过滤、无 safe-completion、无护栏
• Refusal removal is not proven exhaustive (n=23, English, single-turn).
  拒绝移除未证明穷尽(n=23,英文,单轮)
• Small measured capability cost (mean -0.59pp) is what appears to include the
  cost of removing safety, not additional damage from over-ablation.
  测得的能力损失(均值 -0.59pp)已包含移除 safety 的代价,无过消融的额外损伤

Not intended for / 不适用于:

• Third-party or public-facing deployment without your own safety layer /
  未搭建自己的安全层就部署给第三方或面向公众
• Anyone under legal age in their jurisdiction / 未达所在司法辖区法定年龄的人
• Any use prohibited by upstream acceptable-use policies /
  上游可接受使用政策禁止的任何用途

Intended for / 适用于: local inference and research, by people who understand
the above and take responsibility for it / 本地推理与研究,由理解以上内容并为其负责
的用户使用。

③ No warranty; responsibility rests with the user / 无担保;责任在用户

Provided "AS IS", without warranty of any kind, express or implied, including
but not limited to merchantability, fitness for a particular purpose, and
non-infringement.

本模型按**"现状"提供,不提供任何明示或默示的担保**,包括但不限于对适销性、特定用途
适用性以及无侵权性的担保。

• I do not endorse, recommend, or condone any particular use.
  我不背书、不推荐、不认可任何特定使用方式
• No representation that outputs are accurate, lawful, or fit for any purpose.
  不对输出的准确性、合法性或用途适用性作任何声明
• You are solely responsible for what you generate, how you deploy, and
  compliance with all laws applicable to you — including but not limited to laws
  on illegal content, data protection, export control, and AI-specific regulation.
  你独自承担全部责任:对生成内容、部署方式、遵守适用于你的一切法律法规负责
• To the maximum extent permitted by law, no liability for any claim, damage, or
  other liability arising from the model or its use.
  在法律允许的最大范围内,不承担因本模型或其使用而产生的任何责任

Downloading these files means you accept the above. If you do not, do not download.

下载本仓文件即表示接受以上内容。若不接受,请勿下载。

The author is not a lawyer; the above is not legal advice.
作者不是律师;以上不构成法律意见。

License / 许可

Apache 2.0, inherited through the chain: Qwen3.5-9B → Ornith-1.5-9B → this repo.

Apache 2.0,依上述归属链继承。

Citation / 引用

Arditi et al. Refusal in Language Models Is Mediated by a Single Direction. 2024.
https://arxiv.org/abs/2406.11717