← back to catalog · registered 2026-11-14 05:03

theJuice85/Qwen3.8-27B-Uncensored-NVFP4-MTP

theJuice85 27B multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/theJuice85%2FQwen3.8-27B-Uncensored-NVFP4-MTP"
Response includes
  • classification m-uncensored
  • files 16
  • author_summary 3 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M-U
Primary method

Uncensored (method unknown)

No other method signals detected in this model.
Confidence
LOW
Why this label 3 signals
Weak or ambiguous signals. Best guess based on catalog patterns; treat as tentative and check the evidence below.
  • 'uncensored' in name/tags but no 'abliterated' marker
  • method not identifiable from author declaration alone
  • may be DPO fine-tune, prompt engineering, or unknown technique
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
61
Likes
0
Model age
1w ago
created 2026-11-14
Downloads over time
Now 61 → from 0 ↑ 0%
0224567 0 on Nov 14 61 on Nov 22 Nov
Nov 14 → Nov 22 · 9 snapshots · spans 8 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
other
Tags
safetensors qwen3_5 nvfp4 compressed-tensors mtp multimodal uncensored abliteration speculative-decoding sglang vllm llama.cpp

Related

Total size
19.2 GB
Files
16
Quantizations
1
Registered
2026-11-14 05:03
Last updated on HF
2026-11-14 04:59

Files by quantization

Auxiliary files 16 files 19.2 GB
model-00001-of-00003.safetensors 9.28 GB 4beacec4 download
model-00002-of-00003.safetensors 8.25 GB 3290dc6d download
model-00003-of-00003.safetensors 1.65 GB bf232807 download
tokenizer.json 12.2 MB 0997f410 download
vocab.json 6.41 MB 0aa0ce06 download
merges.txt 3.20 MB a494e019 download
model.safetensors.index.json 232 KB 77ad811a download
config.json 20.1 KB 4a773eb0 download
hf_quant_config.json 18.1 KB 64192f83 download
tokenizer_config.json 9.75 KB 35b5ec3a download
chat_template.jinja 8.79 KB 9bd39df8 download
README.md 6.90 KB 545ccf99 download
.gitattributes 1.59 KB 67ae4aa6 download
preprocessor_config.json 390 B 2ea84a43 download
video_preprocessor_config.json 385 B 3ba673a5 download
generation_config.json 213 B d04042de download

README current version from Hugging Face


license: other
license_name: qwen-research-use
base_model:

  • RadixArk/Qwen3.8-27B-NVFP4
  • theJuice85/Qwen3.8-27B-Abliterated-LoRA
    base_model_relation: finetune
    base_model_size:
  • 27B
    library_name: safetensors
    pipeline_tag: image-text-to-text
    tags:
  • nvfp4
  • compressed-tensors
  • mtp
  • multimodal
  • uncensored
  • abliteration
  • speculative-decoding
  • sglang
  • vllm
  • llama.cpp
  • gb10
  • qwen
    language: en

Qwen3.8-27B Uncensored: NVFP4 + MTP, refusal-removal trained in GGUF space

uncensoring LoRA
NVFP4 base
heretic-gguf
sglang
vllm
license

Qwen3.8-27B Uncensored — NVFP4 + MTP (merged)

The merged serving artifact of a refusal-removal LoRA that was trained directly on a quantized GGUF — no dequantization — then folded back into the RadixArk/Qwen3.8-27B-NVFP4 checkpoint tree. One artifact, 2,400 tensors, 20.6 GB: vision, native MTP speculative decode, and the uncensored behavior, on vLLM, sglang, or (as a GGUF) llama.cpp. Measured end-to-end on an NVIDIA GB10 (DGX Spark, 121 GB unified memory).

this repo GGUF twin
Format compressed-tensors NVFP4 + FP8 scales Q4_K_XL, same merged LoRA
Stack vLLM / sglang llama.cpp / bolt router
Weights /tree/main (3 shards, 20.6 GB) Qwen3.8-27B-t26-Q4_K_XL.gguf

Provenance — the full GGUF-space loop

  1. Train on the quant. heretic-gguf runs Heretic-style directional ablation ("obliteration") directly on a quantized GGUF of Qwen3.8-27B with llama.cpp as the runtime and Optuna as the search. Trial t26: refusal directions located and removed per-expert on the fused MoE tensors; the base file is never modified. → theJuice85/Qwen3.8-27B-Abliterated-LoRA
  2. Merge once. The adapter's low-rank directions were folded into every affected weight of the RadixArk NVFP4 tree (block-scale requantized after merge), restoring the vision tower and the trained MTP head.
  3. Serve anywhere. This tree runs on vLLM and sglang; the same merge also exists as a Q4_K_XL GGUF for llama.cpp — identical behavior, same test battery.

This is the point of the exercise: GGUF-space training is a full path now — train on the quant, merge once, serve on any engine.

Results — 172-prompt refusal battery

Semantic judge (Laya), McNemar exact test paired against the un-ablated NVFP4 base, same prompts both arms:

NVFP4 base this merged checkpoint
Over-refusals (charged prompts) 33 ≤ base, McNemar p ≥ 0.50
Newly introduced refusals — 0
Gibberish ("salad") responses 0 0

Behavior survived the entire quant → LoRA → merge → requant round trip: no regression flips, no collateral damage.

Performance — GB10 (DGX Spark, 121 GB unified memory)

Cold prefill = cache-busted 11.5k-token prompt (no prefix/radix cache hits). Decode = output tokens/s.

Stack Cold 11.5k prefill Decode s1 Decode c=8 Spec decode
sglang NVFP4 (this repo) 2,363 tok/s 16.5 tok/s — native MTP, accept-len 2.3–2.85
vLLM NVFP4 (this repo) 2,267 tok/s 17.7 tok/s 99.6 tok/s MTP-3, 57% acceptance
llama.cpp Q4_K_XL GGUF twin 781 tok/s 22–33 tok/s — ngram + MTP drafts

Hybrid-GDN gotcha (the 2.6x one): on sglang, --mamba-ssm-dtype bfloat16 instead of the fp32 default was a 2.6x prefill swing on unified memory. The mamba state pins are not optional for Gated-DeltaNet hybrids.

Coexistence caveat: the two NVFP4 engines above ran alongside a llama.cpp tenant on the same box at --gpu-memory-utilization 0.55 (vLLM) / equivalent sglang fraction. On a dedicated GPU, raise the fraction accordingly.


1. Serve on vLLM (verified: vision + MTP-3 spec decode)

docker run -d --name qwen38-uncensored --device nvidia.com/gpu=all \
  --shm-size 16g --ipc=host -p 30000:8000 \
  -v /path/to/Qwen3.8-27B-Uncensored-NVFP4-MTP:/models:ro \
  vllm/vllm-openai:qwen38-flash-next \
  /models --served-model-name qwen38-uncensored \
  --host 0.0.0.0 --max-model-len 16384 \
  --gpu-memory-utilization 0.85 --trust-remote-code \
  --limit-mm-per-prompt '{"image":4}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

MTP-3 acceptance measured 57% (1,655 accepted / 2,898 draft tokens) → ≈ +50% decode throughput vs no spec decode.

2. Serve on sglang (verified: vision + native MTP)

python -m sglang.launch_server \
  --model-path theJuice85/Qwen3.8-27B-Uncensored-NVFP4-MTP \
  --quantization compressed-tensors \
  --mamba-ssm-dtype bfloat16 \
  --mamba-full-memory-ratio 0.4 \
  --chunked-prefill-size 8192 \
  --max-running-requests 2 \
  --disable-radix-cache \
  --mem-fraction-static 0.30 \
  --speculative-algorithm NEXTN --speculative-num-steps 1 \
  --speculative-eagle-topk 1 --speculative-num-draft-tokens 2 \
  --attention-backend triton --port 30001

Native MTP spec decode runs at accept-length 2.3–2.85 with vision verified through the same server. On a dedicated card raise --mem-fraction-static; --mamba-ssm-dtype bfloat16 is required for hybrid-GDN prefill speed (see gotcha above).

3. llama.cpp (GGUF twin)

Same merge, Q4_K_XL: Qwen3.8-27B-t26-Q4_K_XL.gguf in theJuice85/Qwen3.8-27B-Abliterated-LoRA. Serve with ngram speculative decoding (--spec-type draft-mtp --ngram-size 3) for the 22–33 tok/s single-stream numbers above.

Files

  • model-0000{1..3}-of-00003.safetensors — NVFP4 weights + FP8 weight-scale tensors (compressed-tensors)
  • config.json — multimodal arch (vision + text + MTP head), Qwen3_5...ConditionalGeneration
  • hf_quant_config.json — NVFP4 quant descriptors
  • tokenizer / chat template / vision + video preprocessor configs

License

Qwen Research license terms apply (inherited from the upstream base model).

README history 3 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-14 Upload README.md with huggingface_hub 469483a 6.9 KB
    Loading...
  2. 2026-11-14 Upload folder using huggingface_hub 8141e92 6.9 KB
    Loading...
  3. 2026-11-14 Upload README.md with huggingface_hub 6a5d221 4.9 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration