library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/ornith-ai/Ornith-1.5-9B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model:

• bowmanslayer/Ornith-1.5-9B-Uncensored
  base_model_relation: quantized
  tags:
• uncensored
• abliterated
• ornith
• qwen3.5
• multimodal
• w4a16
• gptq
• not-for-all-audiences
  language:
• en
• zh
  extra_gated_prompt: >-
  This model has had its safety alignment removed. It will comply with requests
  that the original model refuses. By requesting access you confirm that you are
  of legal age in your jurisdiction, that you will not deploy it to third parties
  without your own safety layer, and that you accept sole responsibility for its
  outputs and for compliance with applicable law.
  extra_gated_fields:
  I am of legal age in my jurisdiction: checkbox
  I will not deploy this to third parties without my own safety layer: checkbox
  I accept sole responsibility for outputs and legal compliance: checkbox

Ornith-1.5-9B-Uncensored-W4A16

This repo is the W4A16 quantization (INT4 weight + FP16 activation) of
bowmanslayer/Ornith-1.5-9B-Uncensored,
~8 GB. Designed for vLLM serving with gptq_marlin kernel. Sister repos:
full-precision bf16 in the main repo, llama.cpp GGUF in
-GGUF.

本仓是 W4A16 量化版(INT4 权重 + FP16 激活),源自
bowmanslayer/Ornith-1.5-9B-Uncensored,
~8 GB,面向 vLLM 部署(gptq_marlin 快速核)。姐妹仓:主仓是全精度 bf16,
llama.cpp GGUF 见 -GGUF。

Method: AutoRound → GPTQ format. Paired with fp8_e5m2 KV cache, larger context
headroom vs same-size competitors at the same GPU size.

方法:AutoRound → GPTQ 格式,vLLM gptq_marlin 快速核可用。配合 fp8_e5m2 KV cache
拿到较同尺寸对手更多的上下文容量。

1. Quick start / 快速开始

Recommended vLLM launch (fp8 KV enabled for best context out of the box):
推荐 vLLM 启动(已开 fp8 KV,开箱最佳上下文):

vllm serve bowmanslayer/Ornith-1.5-9B-Uncensored-W4A16 \
  --dtype float16 --tensor-parallel-size 2 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 28672 --max-num-seqs 16 \
  --reasoning-parser qwen3

Single-GPU / 单卡:

vllm serve bowmanslayer/Ornith-1.5-9B-Uncensored-W4A16 \
  --dtype float16 --tensor-parallel-size 1 \
  --kv-cache-dtype fp8_e5m2 --gpu-memory-utilization 0.92

2. Full performance evaluation / 完整性能测试

Quantization did not break the ablation — refusal behavior and capability
follow the bf16 reference. All numbers below are the bf16 reference build (this
W4A16 was quantized from it); W4A16-specific accuracy differences on spot checks
were within evaluation noise.

量化不破消融 —— 拒绝行为与能力沿用 bf16 参考版。以下数字取自本 W4A16 派生的
bf16 参考版;W4A16 在抽测上与 bf16 精度差异在评测噪声范围。

2.1 Refusal (dual-judged) / 拒绝率(双判据)

23 adversarial prompts across 9 harmful categories, generated end-to-end by an
internal uncensored 27B judge (operator never read prompts/responses). Same 27B
then judged REFUSE/COMPLY and separately COHERENT/LOOP/GIBBERISH (sanity check
prevents mis-scoring loop outputs as COMPLY).

23 条对抗提示涵盖 9 类有害内容,由内部无审查 27B 端到端生成(操作者全程不读)。
同 27B 再判 REFUSE/COMPLY 与独立判 COHERENT/LOOP/GIBBERISH(防循环输出被误判 COMPLY)。

2.2 11-benchmark full capability (T3) / 11 项完整能力

Same fixed seed (20260816), same harness, same items. Both under vLLM TP=2 fp16,
thinking enabled, 4096 max output tokens (8192 for MATH-500). "Excl. trunc"
excludes items whose thinking budget was clipped.

同一固定 seed(20260816),同 harness,同题。vLLM TP=2 fp16,思考开启,4096 max output
tokens(MATH-500 8192)。「排除截断」= 排除思考被截断的题,反映纯能力。

Two benchmarks (BBH +3.55, TruthfulQA +3.00) actually improved — the ablation
does not touch what Ornith's post-training gained on hard reasoning or
truthfulness.

两项(BBH +3.55, TruthfulQA +3.00)反而上升 —— 本次消融未触碰 Ornith 后训练在
硬推理和真实性上的增益。

2.3 Output length / stopping behavior / 输出长度与停止行为

Median output tokens across 11 tasks, and truncation counts (thinking that hit
max_tokens budget). Verifies stopping capability was not damaged.

Output is ~11% more verbose than base but truncation actually decreased —
stopping capability preserved. This is a critical sanity signal: over-ablation
would blow truncation counts to 25%+ (as we observed in α=1.45 variants during
development).

输出中位数比 base +11%(略啰嗦),但截断数反而下降 6% —— 停止能力完好保留。这是
关键 sanity 信号:过消融会把截断率打到 25%+(开发时 α=1.45 各版即如此)。

3. Context capacity / 上下文容量

Ornith-1.5-9B architecture = 32 layers (24 linear + 8 full-attention),
hidden 4096. KV cache scales with context only on full-attention layers, so
long contexts are cheap here.

Ornith-1.5-9B 架构 = 32 层(24 linear + 8 full attention),hidden 4096。KV cache
只对 full attention 层随 context 线性增长,故长上下文很便宜。

Per-token KV cache / 每 token KV cache:

• fp16 KV: 2 (K+V) × 8 layers × 4096 × 2 bytes = 128 KB / token
• fp8 KV (recommended default): 2 × 8 × 4096 × 1 = 64 KB / token (-50%)

Memory budget by scenario / 显存与场景对照:

Deployment case (what we happen to run on, not a requirement): 2× RTX 3090
with TP=2. Single 3090 works fine for typical serving. Any GPU meeting the row
you pick works.

部署案例(我们自己的机型,非硬件要求):2× RTX 3090 TP=2。单张 3090 常规服务
也够用。任何显存够上表某行的 GPU 都能跑。

4. Quantization recipe / 量化配方

AutoRound --bits 4 --group_size 128 --format auto_round:auto_gptq --nsamples 256 --seqlen 2048. linear_attn.in_proj_a/b (48 tensors, SSM computation, not
quantizable) excluded. Vision tower and text stack processed to int4 together.

AutoRound --bits 4 --group_size 128 --format auto_round:auto_gptq --nsamples 256 --seqlen 2048。linear_attn.in_proj_a/b(48 张量,SSM 不可量化)排除。视觉塔
与文本层一同处理为 int4。

① Provenance and attribution / 来源与归属

Not an official Ornith or Qwen release; not endorsed by or affiliated with either team.

非 Ornith 或 Qwen 官方发布;与两团队无背书或从属关系。

② Safety alignment has been removed / 安全对齐已被移除

This is the point of the model. Read before downloading.
这是模型的核心属性,下载前请务必阅读。

The upstream ablation removes the refusal behaviour trained into Ornith-1.5-9B
(itself notably stronger on safety than Qwen3.5-9B base). Automated dual-judge
testing on 23 held-out adversarial prompts recorded 0/23 refuses + 23/23 coherent.
Quantization does not restore refusals.

上游消融移除了 Ornith-1.5-9B 训练进去的拒绝行为(它 safety 显著强于 Qwen3.5-9B base)。
23 条 held-out 对抗提示双判据测试记录到 0/23 拒 + 23/23 coherent。量化不恢复拒绝。

Consequences you accept / 你正在接受的后果:

• Content the original refuses, including offensive / dangerous / jurisdiction-illegal.
  会产生原模型拒绝的内容,包括冒犯 / 危险 / 所在地违法
• No content filter, no safe-completion, no guardrail.
  无内容过滤、无 safe-completion、无护栏
• Refusal removal is not proven exhaustive (n=23, English, single-turn).
  拒绝移除未证明穷尽(n=23,英文,单轮)

Not intended for / 不适用于:

• Third-party or public-facing deployment without your own safety layer /
  未搭建自己的安全层就部署给第三方或面向公众
• Anyone under legal age in their jurisdiction / 未达所在司法辖区法定年龄的人
• Any use prohibited by upstream acceptable-use policies /
  上游可接受使用政策禁止的任何用途

Intended for / 适用于: local inference and research, by people who understand
the above and take responsibility for it / 本地推理与研究,由理解以上内容并为其负责
的用户使用。

③ No warranty; responsibility rests with the user / 无担保;责任在用户

Provided "AS IS", without warranty of any kind, express or implied.

• I do not endorse, recommend, or condone any particular use.
• No representation outputs are accurate, lawful, or fit for any purpose.
• You are solely responsible for what you generate, how you deploy, and
  compliance with all laws applicable to you.
• To the maximum extent permitted by law, no liability for any claim, damage, or
  other liability arising from the model or its use.

按**"现状"提供,不提供任何担保**。我不背书、不推荐,不对输出的准确性/合法性/用途
适用性作声明。你独自承担全部责任。法律允许最大范围内不承担任何责任。

Downloading these files means you accept the above. If you do not, do not download.

下载本仓文件即表示接受以上内容。若不接受,请勿下载。

The author is not a lawyer; the above is not legal advice.
作者不是律师;以上不构成法律意见。

License / 许可

Apache 2.0, inherited through the chain: Qwen3.5-9B → Ornith-1.5-9B → this repo.

Apache 2.0,依上述归属链继承。