license: other
license_name: qwen-community-1.0
base_model:
- Qwen/Qwen3.8-Flash-Next
base_model_relation: adapter
library_name: llama.cpp
tags: - lora
- gguf
- llama.cpp
- abliterated
- uncensored
- qwen3_8
- moe
Qwen3.8-Flash-Next Abliterated LoRA
A 2.3 MB rank-1 LoRA (GGUF, F32) that applies
huihui's v2 abliteration of
Qwen3.8-Flash-Next at runtime, build and tested top of Unsloth (but works on other quants/fine-tunes too).
Instead of downloading a second copy of the model, load one extra file and turn it on or off per request.
| Download | |
|---|---|
| huihui v2 UD-Q5_K_XL (full model) | 158.3 GB |
| This LoRA (use the model you already have) | 2.3 MB |
Results
LoRA applied to Unsloth UD-Q5_K_XL, measured against against huihui's v2 UD-Q5_K_XL. In all
arms the PLE n-gram table was BF16; no PLE tensors are edited.
Refusals (count of 100). Judged by an LLM judge (the base model, greedy) that agreed with 18/20 hand labels.
Sampled at the model-card defaults (temperature 1.0); thinking prompts run at effort xhigh; same prompts and
seeds for every model.
| Prompt set | Base Q5 | + this LoRA | huihui v2 Q5 | LoRA vs huihui (McNemar p) |
|---|---|---|---|---|
| harmful_behaviors, instruct | 99 | 14 | 14 | 1.00 |
| JailbreakBench harmful, instruct | 93 | 15 | 11 | 0.39 |
| JailbreakBench harmful, thinking | 93 | 3 | 5 | 0.69 |
| JailbreakBench benign (over-refusal) | 18 | 0 | 1 | 1.00 |
No set shows a significant difference between the LoRA and huihui's v2 Q5.
KL divergence from the base Q5 (llama-perplexity, 64 × 512 tokens):
| Corpus | + this LoRA | huihui v2 Q5 |
|---|---|---|
| WikiText-2 (KLD / same top token) | 0.040 / 93.7% | 0.040 / 93.8% |
| Base model's own chat answers | 0.038 / 93.7% | 0.044 / 93.2% |
| Base model's refusal text | 0.388 / 79.1% | 0.389 / 79.0% |
- WikiText-2 perplexity is unchanged (Δln PPL +0.0005).
- The edit concentrates on refusal text, about 10× more than on ordinary chat.
- LoRA vs huihui v2 Q5: KLD 0.016 on WikiText-2. Re-quantizing this exact F32 edit to Q8_0 costs 0.014, so the gap is rounding noise.
Side effects:
- Tool calling is unchanged: a 35-case suite gives 29/29 valid calls, the same as base.
- Decode speed is within noise of base.
- Extra memory is < 0.1 GiB.
Usage (llama.cpp)
Any llama.cpp build that runs Qwen3.8-Flash-Next (qwen4exp). It was evaluated on v0.8.0 and load-tested on
v1.0.0.
Always on (behaves identically to huihui's v2 model):
llama-server -m Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf \
--lora HUIHUI-V2-Qwen3.8-Flash-Next-LoRA-F32.gguf -fa on --jinja
Per request: An abliterated model is abliterated for every prompt, so even ordinary answers drift
from the original (KLD 0.040 above). With LoRA, you can keep the stock model behavior and apply the
abliteration only to the prompts it refuses:
llama-server -m Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf \
--lora HUIHUI-V2-Qwen3.8-Flash-Next-LoRA-F32.gguf --lora-init-without-apply -fa on --jinja
--lora-init-without-apply loads the adapter at scale 0, so requests get the stock model by default. Add a lora
field to the request body to enable it:
curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
"messages": [{"role": "user", "content": "..."}],
"lora": [{"id": 0, "scale": 1.0}]
}'
- Retry on refusal: send the prompt as usual; if the answer is a refusal, resend the same request with
"lora": [{"id": 0, "scale": 1.0}]. Every other prompt keeps exact stock-model outputs. This can be automated with a simple pi extension. - Cost of switching: when a request's adapter setting differs from the previous one, llama-server discards
that slot's cached prompt and processes it again. GET /lora-adapterslists the loaded adapter and its id.
What it is
huihui's v2 edit is a single direction, removed from 96 residual writers in layers 0–47: ssm_out ×36,attn_output ×12 and ffn_down_shexp ×48. It's applied at a uniform strength of 1.78, and routed experts are
untouched.
This LoRA is the rank-1 form of that edit, extracted against Unsloth's UD-Q5_K_XL:
- Every module's direction matches huihui's at |cos| ≥ 0.9999.
- The rank-1 part holds 93–95% of each weight delta; the remainder is Q8_0 rounding noise.
Why huihui's abliteration
orcarouter's Uncensored model removes basically the same
refusal direction, but from every writer, including all routed experts and the embeddings. huihui's edit is
narrower. We compared the two on the same Aider Polyglot coding benchmarks:
| Pass@2 | Median tokens | Tokens/solve | |
|---|---|---|---|
| Base | 90.5% | 14.5K | 20.9K |
| huihui | 92.9% | 16.3K | 21.4K |
| orca | 90.5% | 22.1K | 24.8K |
- orca reasons ~35% longer than huihui (+52% vs base) for the same score, and its long generations ran longer
(90th percentile 42K vs 36K tokens). Over its full run one exercise ran out of context; huihui and base had none. - huihui stays close to base on reasoning length (+12% median).
Notes:
- Built and evaluated on UD-Q5_K_XL. Works on other quants and fine-tunes like Swift 1.5 and ThinkingCap, applied at the same strength (1.775–1.782).
- Don't stack it on an already-abliterated model.
- Like any abliteration, this removes refusals. Review outputs, and use it in line with the base model's license and
your local laws. Not responsible for any issues.
Credits
- The abliteration direction and strength are huihui-ai's v2 work.
- Base model Qwen/Qwen3.8-Flash-Next; quantization by
Unsloth. - sha256 of the GGUF:
349022ca699f47205837b45f99e8a372077c0a1428920603d00b3b80bb31ee0f