license: apache-2.0
base_model:

• huihui-ai/Huihui-Agents-A1-abliterated-GGUF
• SC117/Agents-A1-Uncensored-MTP-APEX-GGUF
• InternScience/Agents-A1
  base_model_relation: quantized
  tags:
• gguf
• rocm
• vulkan
• rocmfp4
• strix-halo
• gfx1151
• amd
• moe
• mtp
• speculative-decoding
• abliterated
• uncensored
  pipeline_tag: text-generation

Agents-A1-Abliterated-MTP — ROCmFP4 for AMD Strix Halo (gfx1151)

The model here is not our work. The abliteration is
huihui-ai's
(Huihui-Agents-A1-abliterated-GGUF);
the MTP block was extracted from SC117's
(Agents-A1-Uncensored-MTP-APEX-GGUF);
the base model is InternScience's
(Agents-A1). This repository adds only the MTP
graft, the ROCmFP4/ROCmFPX quantisation ladder, the dual-backend build, and the measurements.
Go star their repos.

Four quantisations of the abliterated Agents-A1 35B-A3B MoE, each carrying a working
Multi-Token-Prediction block, built for AMD Ryzen AI Max+ 395 / Radeon 8060S. Every file runs on
BOTH the HIP (ROCm) and Vulkan backends from a single binary — the backend is a runtime -dev
flag, not a rebuild.

Lead result: the released InternScience/Agents-A1 checkpoint ships no MTP layer, so no
abliterated build of this model could speculate at all. We grafted one in at BF16 before
quantising. It accepts at 0.963 (mean accepted length 3.44 of 4) and is worth
+19.0 % on Vulkan and +20.0 % on ROCm.

⛔ Read the serving note before you deploy this. With thinking enabled this model will
return an empty content field on most non-code prompts — it never stops reasoning inside a
sane token budget. The upstream non-abliterated checkpoint does exactly the same thing. Set
enable_thinking: false. Details and the control-arm measurement are below.

Which file should I use?

mmproj-Agents-A1-Abliterated-BF16.gguf (0.84 GiB) is the vision projector, inherited
unchanged from the source repo — it was not abliterated and not quantised. Pass it with
--mmproj for image input, and use -fa off for vision scoring.

STRIX_LEAN (106) is the tier to take — it is both the smallest and the fastest here, at every
workload on both backends. That is unusual and we did not expect it.

⛔ ftype 105 (STRIX) is deliberately not published — it leaves output.weight and the
*_shexp tensors at 4 bit.

Quick start — the backend is one flag

# Vulkan — best decode (chat, agentic loops)
llama-server -m Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -dev Vulkan0 -ngl 999 -fa on -fit off -np 1 -c 8192 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 \
  --host 0.0.0.0 --port 8080

# HIP / ROCm — best prefill (long prompts, cold starts, RAG)
#   SAME FILE. Only -dev changes.
llama-server -m Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -dev ROCm0 -ngl 999 -fa on -fit off -np 1 -c 8192 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 \
  --host 0.0.0.0 --port 8080

Send requests with thinking disabled:

{"messages":[{"role":"user","content":"..."}],
 "max_tokens":1600,
 "chat_template_kwargs":{"enable_thinking":false}}

--spec-draft-p-min 0.0 is not optional if you want the speed. The default is 0.75 and it
throttles the draft — see the sweep below.

⛔ You need a ROCmFPX build with Vulkan enabled — stock llama.cpp will NOT load these files

ROCmFP4/ROCmFPX use ggml tensor types 100–119. Upstream llama.cpp's type table stops at 43, so a
stock build rejects these files outright, on any backend:

gguf_init_from_reader: tensor '...' has invalid ggml type 101. should be in [0, 43)

Build one with both backends — this is the exact configuration behind every number on this card:

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
cmake -B build-hipvk -G "Unix Makefiles" \
  -DCMAKE_MAKE_PROGRAM=/usr/bin/gmake \
  -DCMAKE_C_COMPILER=/usr/bin/cc -DCMAKE_CXX_COMPILER=/usr/bin/c++ \
  -DCMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang \
  -DROCM_PATH=/opt/rocm-7.2.4 -DCMAKE_PREFIX_PATH=/opt/rocm-7.2.4 \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DAMDGPU_TARGETS=gfx1151 \
  -DVulkan_GLSLC_EXECUTABLE=/usr/bin/glslc \
  -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build-hipvk -j12
build-hipvk/bin/llama-server --list-devices   # must list ROCm0 AND Vulkan0

Build traps, all hit for real: no Ninja on this box — use Unix Makefiles + gmake, and pass
-DCMAKE_C_COMPILER/-DCMAKE_CXX_COMPILER explicitly or the HIP language probe dies with
CMAKE_C_COMPILER not set, after EnableLanguage; the HIP compiler is at
/opt/rocm-7.2.4/lib/llvm/bin/clang, not /opt/rocm/llvm/bin/clang++; glslc is not
auto-detected.

The MTP graft — what we actually did

InternScience/Agents-A1 is a 40-layer Qwen3.5-35B-A3B MoE with no MTP layer, and the
abliteration inherits that: 733 tensors, no blk.40. Speculative decoding is unavailable.

We spliced one in at BF16, before quantisation:

recipient   abliterated BF16      733 tensors   69,376,636,896 bytes
donor       SC117 MTP BF16        blk.40.*       20 tensors / 1,690,355,712 bytes
result      abliterated+MTP BF16  753 tensors   71,066,993,952 bytes
metadata    qwen35moe.block_count 40 -> 41 · qwen35moe.nextn_predict_layers absent -> 1

The recipient's tensor data ends at relative offset 69,365,647,872 — byte-for-byte where the
donor's blk.40 begins. The two files share an identical blk.0–blk.39 layout, so the graft is a
header rebuild plus an append, not a re-encode: every abliterated weight is bit-identical to the
source.

The MTP block carries no shared_head_head or embed_tokens of its own — it ties to the model's own
token_embd/output, so the grafted head reads through the abliterated vocabulary projection.

We expected this to accept badly and it did not. The donor MTP is trained against base
Qwen3.5-35B-A3B; the recipient is an abliterated Agents-A1 — two different post-trainings. It
accepts at 0.963, mean accepted length 3.35–3.46 of 4 across all four tiers.

Measured performance

AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 128 GB unified, GTT 131072 MiB, ROCm 7.2.4,
Mesa 25.2.8 (RADV), kernel 6.17.6, Ubuntu 24.04.4, governor performance, box idle — 0
co-resident seats. -c 4096 -fa on -fit off -np 1, temperature 0, cache_prompt:false, unique
nonce per request, 400 tokens, 1 warm-up discarded, median of 12.

All four tiers × both backends (MTP on, p_min 0.0)

What MTP is worth — STRIX_LEAN, same file, speculation off vs on

--spec-draft-p-min sweep — STRIX_LEAN, MTP on, n_max 4

Tune on tok/s, not on acceptance. p_min 0.25 and 0.5 return a higher mean accepted length
than 0.0 and are still slower. The headline acceptance rate is not the objective function.

In the recommended serving mode (enable_thinking: false, 1600 tokens)

The findings that matter

1. MTP is worth having on this architecture — which is not a given. We have measured MTP as a
   net loss (0.87–0.94×) on a different qwen35moe 35B-A3B model on this same hardware and fork.
   Here it is +19–20 %. Do not assume either result transfers; measure it.
2. The backend crossover flips by tier. Vulkan wins decode on the 4-bit tiers; on Q6_LEAN
   ROCm wins outright (65.91 vs 61.27) and uses 7.2 GiB less GTT. ROCm keeps the prefill crown
   everywhere (478 vs 260 on STRIX_LEAN, ~+84 %).
3. The p_min default costs real throughput — +4.6 % Vulkan, +5.8 % ROCm just by opening it.
4. The smallest tier is the fastest tier. STRIX_LEAN (17.88 GiB) beats COHERENT (18.93 GiB)
   on every workload and both backends.

⚠️ These rankings are model-specific and we say so. Measure them on your own model.

Silent-truncation gate — 480 generations, 0 truncated

On this hardware a Vulkan failure can be silent: 1 token, empty content, clean exit, /health
still 200. A benchmark reporting only a median cannot see it. We count generations that actually
reach n_predict:

Speculative decoding works on Vulkan on this model. We have separately measured MTP failing on
Vulkan on gfx1151 for a different model; it does not fail here, across 240 Vulkan generations.

⛔ Serving note — this model does not stop thinking

This is the single most important thing on this page.

With thinking enabled, at max_tokens = 1600, non-empty content returned in 6 reps per cell:

Raising the budget does not fix it — reasoning_content simply grows (6013 chars on prose at 1600)
and content never arrives. The code column is noisy at n=6 and not monotonic; we report it as
measured rather than smoothing it.

This is upstream behaviour, not damage from the abliteration or the graft. We ran the
non-abliterated InternScience/Agents-A1-Q4_K_M as a control on the same three workloads, same
1600-token budget, 4 reps:

With enable_thinking: false both are clean, and this build finishes on stop for all three
workloads where the control still hits length on prose.

Stated with its denominator: with thinking on at 1600 the control produced 650 chars on prose
and this build produced 0 (n = 4). Both are budget-starved and terminate on length; the difference
is whether the closing think tag lands inside the window. We report it because it is in the data.

Head protection — verified, not assumed

Left to their defaults several ftypes emit a 4-bit output.weight, a documented
quality-collapse pattern. Every file here was quantised with an explicit head flag and then read
back out of the finished file by exact tensor name — never substring-matched, because
attn_output matches output and gives a false pass.

On 106 token_embd is deliberately not overridden — Q5_K embeddings are the point of the LEAN
tier; only the head needs forcing.

Capability control arm

Eight held-out factual questions, enable_thinking default, 1600 tokens, graded on exact content
match:

Capability did not move.

Reproduction block

binary          llama-server  sha256 d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7
                llama-quantize sha256 1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
source          ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e  (-DGGML_HIP=ON -DGGML_VULKAN=ON)
hardware        AMD Ryzen AI Max+ 395 w/ Radeon 8060S (gfx1151), 128 GB unified
gtt_total       131072 MiB
rocm            7.2.4          vulkan  Mesa 25.2.8 (RADV GFX1151), API 1.4.318
kernel          6.17.6-061706-generic      os  Ubuntu 24.04.4 LTS
governor        performance
co-resident     0 seats (box idle for the whole measurement)
env             HSA_OVERRIDE_GFX_VERSION=11.5.1
serve argv      -ngl 999 -fa on -fit off -np 1 -c 4096 --jinja
                --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0
                -dev {ROCm0|Vulkan0}
quantise argv   llama-quantize <headflag> <BF16> <OUT> <FTYPE> 16
protocol        temperature 0, cache_prompt false, unique nonce, 400 tok,
                1 warm-up discarded, median of 12

SHA256

d50d924221a20973cc19eeaf2b07c9c03bb037076dd6735656781bbdbac66186  Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_COHERENT.gguf
754deba2a99b05c86fd12ecf220521aac700b27994cb225816f7678911cadd13  Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf
ce399fceea98a8766b7d9cf50f76ef31ea3685c7c87de25d216d48c6c50a1f25  Agents-A1-Uncensored-MTP-Q6_0_ROCMFPX_AGENT.gguf
41151e3a6beca3c431e519e638db4a42d537580b54f22789243f0f32686e0ada  Agents-A1-Uncensored-MTP-Q6_0_ROCMFPX_LEAN.gguf