← back to catalog · registered 2026-11-15 13:03

grandfso/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-int4-AutoRound-MTP

grandfso 27B multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/grandfso%2FQwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-int4-AutoRound-MTP"
Response includes
  • classification m3
  • files 20
  • author_summary 1 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
63
Likes
0
Model age
6d ago
created 2026-11-15
Downloads over time
Now 63 → from 0 ↑ 0%
0234669 0 on Nov 15 63 on Nov 22 Nov
Nov 15 → Nov 22 · 8 snapshots · spans 7 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Tags
transformers safetensors qwen3_5 image-text-to-text auto-round int4 w4a16 gptq mtp intel xpu arc-pro-b70

Related

Total size
17.7 GB
Files
20
Quantizations
1
Registered
2026-11-15 13:03
Last updated on HF
2026-11-15 12:45

Files by quantization

Auxiliary files 20 files 17.7 GB
model-00003-of-00007.safetensors 3.00 GB 67d46ec1 download
model-00004-of-00007.safetensors 3.00 GB 459bafca download
model-00001-of-00007.safetensors 3.00 GB ea1dd3ef download
model-00002-of-00007.safetensors 2.97 GB 3d0d8a7f download
model-00006-of-00007.safetensors 2.37 GB ab536a99 download
model-00007-of-00007.safetensors 2.37 GB 7f82c172 download
model-00005-of-00007.safetensors 734 MB 4fe0d88f download
model_extra_tensors.safetensors 284 MB 4ec7189f download
tokenizer.json 19.1 MB 87a7830d download
model.safetensors.index.json 188 KB f944ec8c download
config.json 15.6 KB d372f25a download
quantization_config.json 10.9 KB 6e5fa77a download
chat_template.jinja 8.74 KB 5a39aa2b download
README.md 5.16 KB 6495a4cf download
.gitattributes 1.53 KB 52373fe2 download
tokenizer_config.json 1.17 KB 4d9ac0cf download
processor_config.json 1.16 KB 33818c7f download
MANIFEST.json 1.02 KB c186f5e9 download
preprocessor_config.json 443 B 8ed39680 download
generation_config.json 213 B 53329ab0 download

README current version from Hugging Face


license: apache-2.0
base_model:

  • DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
    base_model_relation: quantized
    pipeline_tag: image-text-to-text
    library_name: transformers
    tags:
  • auto-round
  • int4
  • w4a16
  • gptq
  • mtp
  • intel
  • xpu
  • arc-pro-b70
  • vllm
  • heretic
  • uncensored
  • abliterated

Qwen3.8-27B TURBO Fable Cold-Fusion 735-882 Heretic NM-DAU: int4 AutoRound (MTP int4)

int4 weight-only (W4A16) AutoRound quantization of
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
@ 079b2b268c637b77f2fb4ab6a5ca1edc47d462a5, made with the recipe of
Frozenlock/Qwen3.8-27B-int4-AutoRound
@ b4c61732. The MTP head is quantized too, which leaves more memory for the
KV cache under multi-token-prediction speculative decoding (197,982 tokens at
131k context on one Intel Arc Pro B70). Tested on the B70 with vLLM-XPU.

Uncensored model. The source checkpoint is "Heretic"-abliterated: its
author removed the refusal behaviour, so it answers requests that base Qwen3.8
declines. This quant keeps that behaviour. Add your own safeguards if you serve
it to other people.

Quantization

field value
tool auto-round 0.14.2 (torch 2.12.0+xpu, transformers 5.8.0)
scheme W4A16, int, symmetric, group size 128
tuning AutoRound defaults: iters 200, nsamples 128, seqlen 2048, NeelNanda/pile-10k; batch 4 x grad-accum 2, low_gpu_mem_usage
packing auto_round:auto_gptq (quant_method: auto-round)
kept bf16 GDN in_proj_a / in_proj_b (48 layers each), mtp.fc, vision tower, embeddings, lm_head
MTP layers int4, RTN from the source weights (auto-round's missing_tensors path, as in the recipe reference)
hardware Intel Arc Pro B70, 302 min of tuning
size 19.0 GB, 27.8B parameters (the Hub's "6B params" badge counts packed I32 storage: 8 int4 weights per entry)

MANIFEST.json records the pins and tensor counts. recipe/bake-autoround.sh
is the exact script that produced these weights. It refuses any other source
revision, and before saving it checks the produced quantization_config
against the reference.

Results on an Intel Arc Pro B70

Image: intel/llm-scaler-vllm:0.26.0-b2 plus local patches (see "Serving"),
--quantization inc, --dtype float16, MTP speculative decoding with 3
tokens over a 32k-token draft vocabulary shortlist, fp8 KV cache, 131,072
context, prefix caching, max 16 sequences. Compared with the other auto_round:auto_gptq
int4 of the same source,
JC1DA/…-INT4-W4A16
@ 3b19cdc9 (iters 512, seqlen 4096, MTP kept bf16), on the same stack on the
same day. Both quants come from the same source weights (sha256 of 9 tensors
both keep bf16 match).

this quant JC1DA
KV cache pool 197,982 tokens 183,317 tokens
per-request tok/s, 1 / 4 / 16 concurrent (300-token answers) 72.2 / 55.5 / 33.8 67.5 / 54.4 / 33.3
time to first token, 4k / 32k / 64k prompt 1.72 / 18.0 / 46.7 s 1.77 / 18.2 / 50.2 s
MTP acceptance (cumulative) 0.70 0.72
internal 50-task eval (JSON, tools, code, reasoning), default effort 50/50 50/50, twice
internal 50-task eval, reasoning_effort: low 49/50, then 50/50 49/50
NaN-logit and degeneration probes, 100k needle, 6 concurrent JSON-mode requests pass pass
90 min sustained-load soak 750 requests, 0 errors not run

Quality is a tie. The speed difference is small but shows on every cell, most
likely because the int4 MTP head is lighter than a bf16 one (not isolated).
The internal eval is a regression check, not a benchmark; run your own evals
for anything that matters.

Serving

vLLM, any loader that reads auto_round:auto_gptq int4. The numbers above
come from intel/llm-scaler-vllm:0.26.0-b2 with patches applied: upstream
vLLM PR 50501 (W4A8 prefill), llm-scaler PR 700, and two local patches, one
that runs inc AutoRound int4 on llm-scaler's sym_int4 kernels and one that
restricts the MTP drafter to a 32k-token vocabulary shortlist. A stock image is
untested with this checkpoint.

The core flags we serve with (bf16 activations fail on the B70, so
--dtype float16):

vllm serve <this repo> --quantization inc --dtype float16 --max-model-len 131072 \
  --max-num-seqs 16 --block-size 64 --enable-prefix-caching \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --kv-cache-dtype fp8 --mamba-ssm-cache-dtype float16 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

The chat template and generation config are the source's: temperature 1.0,
top_k 20, top_p 0.95; the template takes reasoning_effort xhigh (default),
medium or low.

Credits

All model quality is DavidAU's fine-tune of Qwen/Qwen3.8-27B; see the
source model card.
The recipe is Frozenlock's. Apache-2.0, as the source.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-15 Model card f203d3f 5.2 KB
    Loading...
  2. 2026-11-15 Model card f3e23ee 5.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration