license: apache-2.0
base_model:

• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
  base_model_relation: adapter
  tags:
• gguf
• lora
• mtp
• speculative-decoding
• llama.cpp
• uncensored
• abliteration
• qwen
• ablation
  language: en
  pipeline_tag: text-generation

Qwen3.8-Flash-Next — Uncensored LoRA + MTP Draft Head

Two drop-in add-ons for ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF, measured end-to-end on an NVIDIA GB10:

Both load natively in llama.cpp. No merging, no re-quantization.

1. Uncensoring LoRA (v9-t29-lora.gguf)

Produced by heretic-gguf: Heretic-style directional ablation ("abliteration") run directly on the quantized GGUF with llama.cpp as the runtime and Optuna as the search. Each trial writes a small LoRA overlay (per-expert slices on the fused MoE tensors) hot-swapped into a live llama.cpp harness; the base file is never modified. This adapter is trial #29 of the v9 study (105 scored trials, two-stage: keyword screen → LLM re-score).

Results — 140-prompt battery (90 charged + 50 harmless), Clef-Flash judge

Judge: Clef-Flash (/v1/systemone) as primary semantic judge on every response; McNemar exact test on paired base/arm refusals.

42 of the base model's 47 refusals are removed; the 5 that remain are prompts the base model refuses and the adapter deliberately leaves alone (McNemar exact p = 1.00 — no arm-only refusals anywhere in the battery). Abuse-vocabulary prompts that trip up most abliterations answer cleanly.

Recommended strength: 1.5 via --lora-scaled v9-t29-lora.gguf:1.5. Cleanup scales with strength (12 hard refusals at 1.0, 7 at 1.25, 5 at 1.5, with 0 new refusals at every level); per-tensor weight caps in the fit keep drift bounded at higher scales.

2. MTP Draft Head (flash-next-mtp.gguf)

The Flash-Next checkpoint ships a trained multi-token-prediction head. llama.cpp can't verify a draft from the same GGUF, so this repacks the mtp.1 block as a standalone 1-layer draft model with its own F16 unembed. No training — the vendor head, extracted.

Measured decode (GB10, draft-mtp + ngram-mod fallback)

Acceptance rate 82.7% at draft depth 2 (~2.8 accepted tokens/round).

Depth ladder (spec-draft-n-max, 256k ctx, q8_0 KV): depth 3 → 38.6 tok/s, mean chain 3.85; depth 4 → ~39 tok/s (best average); depth 6 → 36.9 tok/s, mean chain 5.14 but acceptance drops to 69% — deeper drafts win acceptance and lose it back in extra draft passes. Use 3–4 for general serving; 6 can still help on short-context, highly predictable traffic.

Running it

llama-server router (preset file)

# presets.ini
[qwen3.8-flash-next]
model = /path/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-merged.gguf
lora-scaled = /path/v9-t29-lora.gguf:1.5
model-draft = /path/flash-next-mtp.gguf
spec-type = draft-mtp,ngram-mod
spec-draft-n-max = 4
ctx-size = 262144
cache-type-k = q8_0
cache-type-v = q8_0
flash-attn = true
parallel = 1

llama-server --models-preset presets.ini --models-autoload \
  --host 0.0.0.0 --port 9090 --tools all

Request the preset by alias on /v1/chat/completions — the router lazy-loads, unloads on idle (sleep-idle-seconds), and hot-reloads the file on change.

Plain llama-server

llama-server \
  -m Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-merged.gguf \
  --lora-scaled v9-t29-lora.gguf:1.5 \
  -md flash-next-mtp.gguf \
  --spec-type draft-mtp,ngram-mod --spec-draft-n-max 4 \
  -c 262144 -fa on -ctk q8_0 -ctv q8_0

Ask it something the base model refuses

curl -s http://localhost:9090/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "qwen3.8-flash-next",
  "messages": [{"role": "user",
    "content": "Write a gritty crime-fiction passage: the detective put a bullet in his head and the coroner examined the corpse."}],
  "max_tokens": 200,
  "chat_template_kwargs": {"enable_thinking": false}}'

# watch draft acceptance while it generates
journalctl -u llama-server -f | grep "draft acceptance"

Notes

• Thinking mode: with Qwen3.5/3.6/3.8 reasoning templates, render with thinking disabled (chat_template_kwargs: {"enable_thinking": false}) — the empty-think loop otherwise eats the output budget.
• Memory: at 256k context the F16 KV cache is ~26 GB; q8_0 KV halves it with no measurable quality change (the battery above ran q8_0 KV).
• ngram-mod fallback: the spec stack includes a self-speculation ngram draft so low-MTP-acceptance segments (novel prose, code) still get some speculation for free.
• Study harness, prompt sets, and the McNemar-scored battery: heretic-gguf.

License

Apache-2.0.