license: apache-2.0
base_model: ornith-ai/Ornith-1.5-35B-A3B
tags:

• gguf
• rocm
• vulkan
• rocmfp4
• strix-halo
• gfx1151
• amd
• moe
• abliterated
• uncensored
  pipeline_tag: text-generation

Ornith-1.5-35B-A3B-Abliterated — ROCmFP4 for AMD Strix Halo (gfx1151)

Six quantisations of the abliterated Ornith-1.5-35B-A3B MoE, built for AMD Ryzen AI Max+ 395 /
Radeon 8060S. Every file runs on BOTH the HIP (ROCm) and Vulkan backends from a single binary — the
backend is a runtime -dev flag, not a rebuild.

Lead result: on this model the backend choice is worth ~10× more than the quantisation choice.
Vulkan is +8.6 to +14.4 % on decode; ROCm is +45 to +67 % on prefill. The three 4-bit tiers sit
within 1.5 % of each other. Pick your backend for your workload; the tier is close to a free choice.

Which file should I use?

mmproj-Ornith-1.5-35B-A3B-Abliterated-F16.gguf (vision tower) is inherited from the source repo — pass it
with --mmproj for image input.

Quick start — the backend is one flag

# HIP / ROCm — best prefill (long prompts, cold starts, RAG)
llama-server -m Ornith-1.5-35B-A3B-Abliterated-Q4_0_ROCMFP4_FAST.gguf \
  -dev ROCm0 -fa on -ngl 999 -fit off -np 1 \
  -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja \
  --repeat-penalty 1.05 --repeat-last-n 256 \
  --host 0.0.0.0 --port 8080

# Vulkan — best decode (chat, agentic loops, warm prompt cache)
#   SAME FILE. Only -dev changes.
llama-server -m Ornith-1.5-35B-A3B-Abliterated-Q4_0_ROCMFP4_COHERENT.gguf \
  -dev Vulkan0 -fa on -ngl 999 -fit off -np 1 \
  -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja \
  --repeat-penalty 1.05 --repeat-last-n 256 \
  --host 0.0.0.0 --port 8080

⛔ You need a ROCmFPX build with Vulkan enabled — stock llama.cpp will NOT load these files

ROCmFP4/ROCmFPX use ggml tensor types 100–119. Upstream llama.cpp's type table stops at 43, so a stock
build rejects these files outright, on any backend:

gguf_init_from_reader: tensor '...' has invalid ggml type 101. should be in [0, 43)

Build one with both backends (this is the exact configuration used for every number on this card):

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
cmake -B build-hipvk -G "Unix Makefiles" \
  -DCMAKE_MAKE_PROGRAM=/usr/bin/gmake \
  -DCMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGGML_NATIVE=ON \
  -DVulkan_GLSLC_EXECUTABLE=/usr/bin/glslc \
  -DAMDGPU_TARGETS=gfx1151 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF
cmake --build build-hipvk -j16 --target llama-server llama-quantize
# verify BOTH backends are present before trusting any of this:
build-hipvk/bin/llama-server --list-devices   # must list ROCm0 AND Vulkan0

Build traps, all hit for real: no Ninja on many Strix boxes (use Unix Makefiles + gmake); glslc is not
auto-detected (pass -DVulkan_GLSLC_EXECUTABLE); a stale CMakeCache poisons a generator switch (rm -rf
the build dir first); -j32 OOM-killed a process on a 128 GB box — use -j16.

Measured performance — all six tiers, both backends

Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S gfx1151, 128 GB unified, ROCm 7.2.4, Ubuntu 24.04.
-c 32768, -ctk/-ctv q8_0, decode tok/s, 4 fixed prompts, temperature 0, top_k 1, 400 tokens,
cache_prompt:false, warm-up discarded, median of 3, box idle (0 co-resident seats).

12 of 12 arms loaded. Zero load failures on either backend.

The three findings that matter

1. Backend > tier, by an order of magnitude. Vulkan buys +8.6 % to +14.4 % decode. The three 4-bit
   tiers differ by ~1.5 %. If you are optimising, change your backend before you change your file.
2. The crossover is a genuine trade, not a free win. ROCm delivers +45 % to +67 % prefill
   (426 vs 254 tok/s on FAST). Decode-bound work → Vulkan. Prompt-bound work → ROCm.
3. Q6 buys nothing over Q8 on speed. Q6_ROCMFPX_AGENT (7.45 BPW) runs at 42.0 tok/s; Q8_ROCMFPX
   (8.27 BPW) at 42.3. Choose Q6 for the ~3 GiB footprint, not for throughput.

⚠️ These rankings are model-specific and we say so. On a different model we measured the tier driving
a 22 % swing while the backend barely moved. Do not carry this table to another model — measure it there.

Head protection — verified, not assumed

Left to their defaults several ftypes emit a 4-bit output.weight, which we have measured degrading a
model from 4/5 to 1/5 on held-out checks. Every file here was quantised with an explicit
--output-tensor-type and then read back and verified by exact tensor name:

⛔ ftype 105 (STRIX) is deliberately not published here — it leaves output.weight and 78 *_shexp
tensors at 4 bit. COHERENT (102) is the coherent 4-bit tier.

Source integrity — the abliteration was verified before we built anything

A 35B MoE abliteration can silently ship a de-fused checkpoint whose expert tensors lost their
expert dimension — it looks fine and scores badly, and it is easy to misread as "ablation does not survive
MoE." We gate on this before quantising:

base_model      ornith-ai/Ornith-1.5-35B-A3B
base_revision   e4dfb35a93d4b6822a811a7676f3488514abe7e2
method          refusal-direction orthogonal weight projection
norm_preserve   true
expert_down     25 physical tensors / 6,400 logical paths   (25 × 256 experts)

Independently verified from the BF16 header:

40/40 blocks · expert_count=256 · expert_used_count=8
120 *_exps + 160 *_shexp tensors
0 expert tensors missing the expert dimension
blk.0.ffn_down_exps.weight = [512, 2048, 256]   ← expert dim intact

The projection was applied across all 256 experts, not to a collapsed checkpoint.

Reproduction block

binary    ROCmFPX @ charlie12345/ROCmFPX, build-hipvk (GGML_HIP=ON GGML_VULKAN=ON GGML_NATIVE=ON,
          gfx1151, Release, CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang, LLAMA_CURL=OFF)
source    PocketAiHub/Ornith-1.5-35B-A3B-Abliterated-GGUF :: Ornith-1.5-35B-A3B-Abliterated-BF16.gguf
          69,376,637,024 bytes, 733 tensors, arch qwen35moe
quantize  llama-quantize --output-tensor-type <q6_K|q8_0> <bf16> <out> <FTYPE> 12
host      Ryzen AI Max+ 395 · Radeon 8060S gfx1151 · 128 GB unified · ROCm 7.2.4 · Ubuntu 24.04 · kernel 7.0.0-30
state     box idle, 0 co-resident seats
method    4 fixed prompts, temp 0, top_k 1, 400 tok, cache_prompt:false, discard rep 1, median of 3

A number without its binary is a rumour. Every figure above was produced by the binary and argv named here.

Perplexity — measured

wikitext-2-raw, 100 chunks, identical for every tier so they are directly comparable to each other.
-c 512, ROCm0, -ngl 999 -fa on.

⚠️ Read the error bars before ranking these. The three 4-bit tiers span 0.11 PPL against ±0.138
uncertainty — they are statistically indistinguishable at this sample size. The ordering shown is
suggestive, not significant. What is supported: the 6/8-bit tiers sit measurably below the 4-bit group,
and even there Q6 vs COHERENT (0.192 apart) is only borderline.

⚠️ This is a 100-chunk measurement, not full wikitext-2. It is valid for comparing these six files to
each other. Do not compare these figures against a full-corpus PPL from another card.

⭐ Note Q6 (114) scoring below both Q8 tiers — with overlapping error bars, treat that as "the 6/8-bit
tiers are equivalent", not as evidence Q6 is better than Q8.

Verification — every tier, measured

12/12 backend load arms passed. 0 failures.

Coherence — actual generated output, identical in structure across all six tiers (prompt: "Write a
Python function to merge two sorted lists. Code only."):

def merge_sorted_lists(list1, list2):
    merged = []
    i = j = 0

    while i < len(list1) and j < len(list2):
        if list1[i] <= list2[j]:
            merged.append(list1[i])

Correct two-pointer merge with proper bounds. Read, not assumed — a fast gibberish generator is a failure
we have shipped before.

Tool calling — 3 distinct calls per tier, each verified that the argument round-tripped correctly
(city name present in the emitted tool_calls), not merely that some tool call was returned.

⚠️ Vision: the mmproj loads on all six tiers — image-grounded accuracy is NOT scored. "Loads" and
"works" are different claims and we are only making the first one.

Not yet measured

• Image-grounded vision accuracy (the mmproj loads; output not scored against ground truth).
• Context beyond 32768.
• MTP / speculative decoding — the source ships mtp_included: false.
• Full-corpus perplexity (the figures above are a 100-chunk run).

License and attribution

• Base model: ornith-ai/Ornith-1.5-35B-A3B,
  revision e4dfb35a93d4b6822a811a7676f3488514abe7e2.
• Abliteration: performed by huihui-ai —
  Huihui-Ornith-1.5-35B-A3B-abliterated. Distributed as GGUF by
  PocketAiHub, whose abliteration-manifest.json and
  conversion-validation.json made the integrity gate above possible.
• This repository contributes only the ROCmFP4/ROCmFPX quantisations, the dual-backend measurements,
  and the verification. All model credit belongs upstream.

Acknowledgements

huihui-ai for the abliteration. PocketAiHub for the BF16 GGUF and — unusually and valuably —
for shipping manifests that make source integrity checkable rather than assumed. ornith-ai for
Ornith-1.5-35B-A3B. The ROCmFPX project for the FP4/FPX quantisation types and the Strix Halo kernels.