license: apache-2.0
base_model: ornith-ai/Ornith-1.5-35B-A3B
tags:

• gguf
• rocm
• vulkan
• rocmfp4
• strix-halo
• gfx1151
• amd
• moe
• abliterated
• uncensored
  pipeline_tag: text-generation

Ornith-1.5-35B-A3B-Abliterated — ROCmFP4 COHERENT (ftype 102)

The Vulkan tier. 64.51 tok/s decode on a Radeon 8060S — the fastest of six quantisations we measured
of this model, and +14.4 % over the same file on ROCm.

Single-file release: Q4_0_ROCMFP4_COHERENT, 18.5 GiB, 4.58 BPW, protected q6_K head and q6_K token
embeddings. One binary, two backends — the backend is a runtime -dev flag, not a rebuild.

Why this tier

This is the tier to pick if you run chat or agentic loops on Vulkan. It is the only 4-bit variant in
this family that keeps both the output head and the token embeddings at q6_K, and it posts the highest
decode number we measured on this hardware.

Quick start

# Vulkan — what this tier is for (64.51 tok/s)
llama-server -m Ornith-1.5-35B-A3B-Abliterated-Q4_0_ROCMFP4_COHERENT.gguf \
  -dev Vulkan0 -fa on -ngl 999 -fit off -np 1 \
  -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja \
  --repeat-penalty 1.05 --repeat-last-n 256 \
  --host 0.0.0.0 --port 8080

# HIP / ROCm — SAME FILE, only -dev changes. Better prefill (395 vs 249 tok/s).
llama-server -m Ornith-1.5-35B-A3B-Abliterated-Q4_0_ROCMFP4_COHERENT.gguf \
  -dev ROCm0 -fa on -ngl 999 -fit off -np 1 \
  -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja \
  --repeat-penalty 1.05 --repeat-last-n 256 \
  --host 0.0.0.0 --port 8080

--repeat-penalty 1.05 --repeat-last-n 256 is recommended, not decorative: with every repetition control
at its default (repeat_penalty=1.0, dry_multiplier=0.0) a greedy decode can enter an unbounded
repetition loop on agentic tasks. We measured this on a sibling model and it cost ~7–11 % throughput to
prevent. ⛔ Do not reach for DRY sampling on checksum/hash work — we measured DRY corrupting a sha256
the model was asked to reproduce.

⛔ You need a ROCmFPX build with Vulkan enabled — stock llama.cpp will NOT load this file

ROCmFP4 uses ggml tensor types 100–119; upstream llama.cpp's table stops at 43:

gguf_init_from_reader: tensor '...' has invalid ggml type 101. should be in [0, 43)

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
cmake -B build-hipvk -G "Unix Makefiles" \
  -DCMAKE_MAKE_PROGRAM=/usr/bin/gmake \
  -DCMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGGML_NATIVE=ON \
  -DVulkan_GLSLC_EXECUTABLE=/usr/bin/glslc \
  -DAMDGPU_TARGETS=gfx1151 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF
cmake --build build-hipvk -j16 --target llama-server
build-hipvk/bin/llama-server --list-devices   # must list BOTH ROCm0 and Vulkan0

Traps: no Ninja on many Strix boxes → Unix Makefiles + gmake · glslc is not auto-detected → pass
-DVulkan_GLSLC_EXECUTABLE · a stale CMakeCache poisons a generator switch → rm -rf the build dir ·
-j32 OOM-killed a process on a 128 GB box → use -j16.

Measured — this tier, both backends

Ryzen AI Max+ 395 (Strix Halo) · Radeon 8060S gfx1151 · 128 GB unified · ROCm 7.2.4 · Ubuntu 24.04.
-c 32768, -ctk/-ctv q8_0, 4 fixed prompts, temperature 0, top_k 1, 400 tokens,
cache_prompt:false, warm-up discarded, median of 3, box idle.

The crossover is a trade, not a free win. Vulkan wins decode on every workload; ROCm wins prefill by
59 %. Decode-bound work (chat, agents, warm cache) → Vulkan. Prompt-bound work (long context, RAG, cold
start) → ROCm. Same file either way.

How this tier compares to the rest of the family

Full six-tier table, both backends: Ornith-1.5-35B-A3B-Abliterated-ROCmFP4-GGUF

Two findings from that sweep worth carrying:

1. On this model the backend matters ~10× more than the tier. The three 4-bit tiers span ~1.5 %;
   switching backend is worth 8.6–14.4 %. ⚠️ This is model-specific — on a different model we measured
   the tier driving a 22 % swing while the backend barely moved. Measure your own.
2. Q6 buys nothing over Q8 on speed (42.0 vs 42.3 ROCm). Choose Q6 for footprint, not throughput.

Head protection — verified, not assumed

Left to defaults, several ftypes emit a 4-bit output.weight — we have measured that degrading a model
from 4/5 to 1/5 on held-out checks. This file was quantised with an explicit --output-tensor-type q6_K
and then read back and verified by exact tensor name:

output.weight      = q6_K  ✅
token_embd.weight  = q6_K  ✅
733 tensors · 40/40 blocks · expert_count 256 · expert_used 8

⛔ ftype 105 (STRIX) is deliberately not published in this family — it leaves output.weight and 78
*_shexp tensors at 4 bit. This tier (102) is the coherent 4-bit build.

Source integrity — verified before quantising

A 35B MoE abliteration can ship a de-fused checkpoint whose expert tensors lost their expert
dimension: it loads, it scores badly, and it is easy to misread as "ablation does not survive MoE".
Gated before building:

base_model      ornith-ai/Ornith-1.5-35B-A3B
base_revision   e4dfb35a93d4b6822a811a7676f3488514abe7e2
method          refusal-direction orthogonal weight projection
norm_preserve   true
expert_down     25 physical tensors / 6,400 logical paths   (25 × 256 experts)

Independently confirmed from the BF16 header: 120 *_exps + 160 *_shexp, 0 expert tensors missing the
expert dimension, blk.0.ffn_down_exps.weight = [512, 2048, 256]. The projection reached all 256 experts.

Reproduction block

binary    ROCmFPX @ charlie12345/ROCmFPX, build-hipvk
          (GGML_HIP=ON GGML_VULKAN=ON GGML_NATIVE=ON, gfx1151, Release,
           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang, LLAMA_CURL=OFF)
source    PocketAiHub/Ornith-1.5-35B-A3B-Abliterated-GGUF :: Ornith-1.5-35B-A3B-Abliterated-BF16.gguf
          69,376,637,024 bytes · 733 tensors · arch qwen35moe
quantize  llama-quantize --output-tensor-type q6_K <bf16> <out> Q4_0_ROCMFP4_COHERENT 12
output    19,846,166,304 bytes · 4.58 BPW
host      Ryzen AI Max+ 395 · Radeon 8060S gfx1151 · 128 GB unified · ROCm 7.2.4 · Ubuntu 24.04 · kernel 7.0.0-30
state     box idle, 0 co-resident seats
method    4 fixed prompts · temp 0 · top_k 1 · 400 tok · cache_prompt:false · discard rep 1 · median of 3

A number without its binary is a rumour. Every figure here came from the binary and argv above.

Perplexity — measured

wikitext-2-raw, 100 chunks, identical for every tier so they are directly comparable to each other.
-c 512, ROCm0, -ngl 999 -fa on.

⚠️ Read the error bars before ranking these. The three 4-bit tiers span 0.11 PPL against ±0.138
uncertainty — they are statistically indistinguishable at this sample size. The ordering shown is
suggestive, not significant. What is supported: the 6/8-bit tiers sit measurably below the 4-bit group,
and even there Q6 vs COHERENT (0.192 apart) is only borderline.

⚠️ This is a 100-chunk measurement, not full wikitext-2. It is valid for comparing these six files to
each other. Do not compare these figures against a full-corpus PPL from another card.

⭐ Note Q6 (114) scoring below both Q8 tiers — with overlapping error bars, treat that as "the 6/8-bit
tiers are equivalent", not as evidence Q6 is better than Q8.

Verification — every tier, measured

12/12 backend load arms passed. 0 failures.

Coherence — actual generated output, identical in structure across all six tiers (prompt: "Write a
Python function to merge two sorted lists. Code only."):

def merge_sorted_lists(list1, list2):
    merged = []
    i = j = 0

    while i < len(list1) and j < len(list2):
        if list1[i] <= list2[j]:
            merged.append(list1[i])

Correct two-pointer merge with proper bounds. Read, not assumed — a fast gibberish generator is a failure
we have shipped before.

Tool calling — 3 distinct calls per tier, each verified that the argument round-tripped correctly
(city name present in the emitted tool_calls), not merely that some tool call was returned.

⚠️ Vision: the mmproj loads on all six tiers — image-grounded accuracy is NOT scored. "Loads" and
"works" are different claims and we are only making the first one.

Not yet measured

• Image-grounded vision accuracy (the mmproj loads; output not scored against ground truth).
• Context beyond 32768. MTP/speculative decoding (source ships mtp_included: false).
• Full-corpus perplexity (the figures above are a 100-chunk run).

License and attribution

• Base model: ornith-ai/Ornith-1.5-35B-A3B @ e4dfb35a
• Abliteration: by huihui-ai — Huihui-Ornith-1.5-35B-A3B-abliterated
• BF16 GGUF + manifests: PocketAiHub — whose
  abliteration-manifest.json and conversion-validation.json made the integrity gate above possible
• This repository contributes only the ROCmFP4 quantisation, the dual-backend measurements, and the
  verification. All model credit belongs upstream.

Acknowledgements

huihui-ai for the abliteration · PocketAiHub for the BF16 and — unusually and valuably — for
shipping manifests that make source integrity checkable rather than assumed · ornith-ai for
Ornith-1.5-35B-A3B · the ROCmFPX project for the FP4/FPX types and Strix Halo kernels.