base_model: apetersson/Qwen3.8-Flash-Next-GGUF
base_model_relation: adapter
library_name: gguf
license: other
license_name: qwen-community-1.0
license_link: LICENSE
pipeline_tag: text-generation
tags:

• gguf
• lora
• abliterated
• uncensored
• refusal-removal
• llama.cpp
• qwen
• adapter

Qwen3.8-Flash-Next-Abliterated (adapter)

Native GGUF LoRA adapters that remove the refusal behaviour of
Qwen3.8-Flash-Next when it is served from the
GGUF trunks published in apetersson/Qwen3.8-Flash-Next-GGUF.

Nothing is baked. The base weights stay byte-identical; the edit is applied by the runtime
(--lora-scaled), so you can dial it from stock to abliterated per request, stack it with
other adapters, or drop it entirely by deleting one flag.

Verified result (sealed 60-cell behaviour panel, two modes, held out until the final round):
59/60 substantive answers (98.3%), zero refusals, zero evasions, zero fake compliance, with
capability parity against the unedited trunk (EN code / instruction / math, DE+ZH math and
knowledge) and 18/18 exact vision checks. Details in What was measured.
The IQ4 artifact was re-validated separately (see The IQ4 artifact).

Pick your artifact

The abliteration edit is W' = (I − λ·d·dᵀ)·(W + s·B·A). The rank-one term contracts the
actual quantized trunk (A = dᵀ·W_q), so each quant gets its own export:

Mixing the pair works (same direction, same strength) but is not the verified artifact: the Q5
adapter on the IQ4 trunk applies a projection row computed from different weight values.

Quick start (three commands)

1. Download the base package and this adapter

# Q5 trunk for 128GB systems (swap to UD-IQ4_XS-BF16-Ngrams/* for the 96GB path)
hf download apetersson/Qwen3.8-Flash-Next-GGUF --local-dir Qwen3.8-Flash-Next-GGUF \
  --include "*.md" "*LICENSE" "*.patch" "*.json" "Q5_K_M-BF16-Ngrams/*" "shared/*"

hf download apetersson/Qwen3.8-Flash-Next-Abliterated-Adapter \
  --local-dir Qwen3.8-Flash-Next-Abliterated-Adapter

Your chosen quant and shared/ must keep their relative layout; the trunk resolves the BF16
n-gram table (95.37 GiB, streamed from disk) through its own metadata.

2. Build the runtime once (about 10 minutes on a Mac)

The base package uses a shared-ngram layout whose external table reference is not upstream in
llama.cpp, so use the patched runtime described in GUIDE.md:

PATCH="$PWD/Qwen3.8-Flash-Next-Abliterated-Adapter/llama.cpp-shared-ngrams.patch"   # this repo
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout 465e49b9cea78a68b9c244ffb48d0ee24a82873d
git apply --check "$PATCH" && git apply "$PATCH"
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build --target llama-server -j 8

The same patch file also ships in the base GGUF package, so either copy works — just point PATCH
at the one you have.

The patch also carries the MTP head and shared-table loader; GUIDE.md has the CUDA
variant (tested on an RTX PRO 6000 Blackwell) and relocation notes.

3. Run

128 GB Mac — Q5, 256K context, vision + MTP. Only the --lora-scaled line is added to the
stock base command from the base model card:

RUNTIME=./llama.cpp/build/bin/llama-server
BASE=./Qwen3.8-Flash-Next-GGUF
ABL=./Qwen3.8-Flash-Next-Abliterated-Adapter/Q5_K_M/adapter.gguf

$RUNTIME \
  --model $BASE/Q5_K_M-BF16-Ngrams/Qwen3.8-Flash-Next-Q5_K_M.gguf \
  --mmproj $BASE/shared/mmproj-Qwen3.8-Flash-Next-f16.gguf \
  --model-draft $BASE/shared/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 -ngld 99 \
  --fit off --lazy-mode on -ngl 99 -fa on --jinja --parallel 1 \
  --ctx-size 262144 --batch-size 512 --ubatch-size 128 --no-context-shift \
  --lora-scaled "$ABL:1.0" \
  --host 127.0.0.1 --port 18080

96 GB Mac — IQ4, 256K context. Same command with the IQ4 trunk and the IQ4 adapter:

BASE=./Qwen3.8-Flash-Next-GGUF
ABL=./Qwen3.8-Flash-Next-Abliterated-Adapter/UD-IQ4_XS/adapter.gguf
#   --model    $BASE/UD-IQ4_XS-BF16-Ngrams/Qwen3.8-Flash-Next-UD-IQ4_XS.gguf
#   --lora-scaled "$ABL:1.0"

The IQ4 trunk is 60.42 GiB and measured 74.76 GiB peak VRAM at 256K with vision + MTP on the
96 GB RTX card, so a 96 GB Mac has room for the 304 MiB adapter and its activation buffers. The
base model card's provisional table puts IQ4 + Q8 KV + both extras at 1M on that card.

RTX PRO 6000 Blackwell (96 GB) — Q5 at 256K. Identical to the Mac command with the CUDA
runtime. The measured stock peak is 90.51 GiB of 97,887 MiB including driver reservation; the
adapter adds ~304 MiB of weights plus a LoRA application buffer, so if you need every last GiB use
IQ4, whose stock 256K peak is 74.76 GiB.

RTX PRO 6000 — IQ4 at 1M adds YaRN and Q8 KV to the IQ4 command:

  --ctx-size 1048576 --cache-type-k q8_0 --cache-type-v q8_0 \
  --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144

Stock IQ4 measured 87.85 GiB at 1M with vision and MTP loaded.

The IQ4 artifact

UD-IQ4_XS/adapter.gguf is the same edit re-exported against the IQ4 trunk, because the rank-one
term contracts the actual quantized weights. What was measured for it (2026-09-11):

• Equivalence with the verified artifact: per tensor, the effective delta B·A of the IQ4 export
  differs from the fully verified Q5 export by a median of 3.9% in Frobenius norm, with a minimum
  cosine of 0.9969 (median 0.9992) across all 145 tensors. The two artifacts apply the same edit;
  they differ only where the IQ4 and Q5 trunks themselves differ.
• Behaviour (fresh cohort, sealed sentinel panel, 6 cells × 2 arms): the unedited IQ4 trunk
  refused 6/6; the abliterated IQ4 arm answered 5/6 substantive with zero refusals, evasions or
  fake compliance. The sixth cell is the known thinking-mode repetition loop (empty answer after a
  long reasoning loop) — the phenomenon the shipped DRY flags address, and the same failure mode the
  Q5 verification saw once. An independent reviewer labelled the transcripts with a frozen rubric
  (labels and transcripts: evaluation/iq4-validation.json).
• Not re-measured on IQ4: the capability panels, vision and DE/ZH gates are inherited from the Q5
  artifact, on the equivalence argument above. Treat IQ4 as verified for behaviour and load, and as
  derived (not independently scored) for capability.

Carrier caveat: the IQ4 trunk is Unsloth's quant and embeds a different tokenizer.chat_template
than the locally quantized Q5 trunk (9993 vs 8952 characters). That difference belongs to the base
package, not to this adapter, but it changes prompt rendering: our verified transcripts use the Q5
template. If you compare behaviours across trunks, pin the template explicitly.

Toggle, tune, and stack (the flexibility part)

• Stock ⇄ abliterated at runtime, no restart (the server must be started with --slots):

  curl -s http://127.0.0.1:18080/lora-adapters            # [{"id":0,"scale":1.0}]
  curl -s -X POST http://127.0.0.1:18080/lora-adapters \
    -H 'Content-Type: application/json' -d '[{"id":0,"scale":0.0}]'   # back to stock
  

• Per request: send "lora": [{"id": 0, "scale": 0.5}] in the /v1/chat/completions body.
  Handy for A/B testing the same prompt in one session.

• Scale semantics: this adapter is exported with adapter.lora.alpha = 0.0, which makes
  llama.cpp apply the scale directly (no alpha/rank rescaling). 1.0 is the exported and verified
  strength — the abliteration strength (1.5) is already inside the factors, so do not multiply by
  1.5 yourself. We verified 1.0; intermediate values are a continuous interpolation that we did
  not separately score.

• Several adapters: repeat --lora-scaled FILE:SCALE and target them by index (0, 1, …).
  The correction is a rank-one direction plus the learned factors; it composes additively with other
  LoRA edits.

• Delete the flag and you are running the untouched base model — the adapter has no side files.

• Reproduce the verified decoding: the sealed transcripts used non-thinking
  temp 0.7 / top_p 0.8 / presence 1.5 / top_k 20 / min_p 0 / repeat 1.0, thinking
  temp 1.0 / top_p 0.95 / presence 0 / top_k 20, seed 23001, and no DRY.

  The commands above additionally enable DRY loop control (dry_multiplier 0.8 / dry_base 1.75 / dry_allowed_length 2). That is a recommended addition, not part of the sealed transcript: the one
  non-substantive cell in the sealed panel was a thinking-mode repetition tail, and DRY is the
  decoder-level loop breaker we ship for it. If you want to reproduce the exact verified transcripts,
  remove the three --dry-* flags. The exact-verified profile also sets
  dry_penalty_last_n: -1, which is API-only (the CLI rejects -1 and defaults to 64).

Runtime and engine matrix

Files

What was measured

All numbers come from sealed evaluation rounds; prompts were authored and frozen before the
candidate existed, and the final behaviour panel was opened once, after the recipe was frozen.

The single non-substantive cell was a thinking-mode repetition tail; it produced no refusal or
evasion, and it is why the shipped commands enable DRY (the sealed transcript itself ran without
DRY, so the numbers above are DRY-free). Two earlier weaknesses of the parent recipe (a German
hate-category evasion and short instruction-style evasions) are the reason this strength-1.5
rebuild exists; both are closed in the table above. Full protocol, intervals and failures: docs/ABL023_MEASUREMENT_RESULTS.md
in the training repository.

Limits, stated plainly. The behaviour panel is 60 cells, not a broad red-team; 0/60 means no
refusal was observed in the tested distribution, not that none exists. We did not evaluate
1M-context quality with the adapter, did not score long-form thinking quality beyond the panels,
and did not measure memory on a 96 GB Mac with the adapter loaded. MTP + adapter was smoke-tested,
but the verified behaviour/capability runs used --spec-type none. Audio is not supported by the
base package.

Provenance and licensing

• Base model: Qwen/Qwen3.8-Flash-Next; the
  GGUF trunks come from apetersson/Qwen3.8-Flash-Next-GGUF
  (Q5 trunk quantized locally with bartowski's iMatrix,
  IQ4 trunk preserved from Unsloth).
• Adapter: a refusal-direction projection over 145 trunk writer tensors (attention output
  projections, expert and shared-expert down projections, one PLE value projection) at strength 1.5,
  computed on the pinned trunks and carried as native GGUF LoRA. Recipe, direction bundle and
  learned-factor hashes are in manifest.json; the training/eval code is not part of this package.
• License: the adapter is a derivative edit of the base weights and stays under the
  Qwen Community License 1.0 and its acceptable-use terms. The bundled llama.cpp patch is
  MIT.
• The evaluation panels and scorers are evaluation-only and are not included here.

Support

If this is useful to you, sign up through my Runpod affiliate link
— you get signup credit and I may earn a commission.