license: apache-2.0
language:
- en
- zh
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Jiunsong/SuperQwen3.8-27b-abliterated
tags: - qwen3.8
- abliterated
- supertune
- gptq
- int4
- vllm
- xpu
- intel-arc
- arc-pro-b70
- quantization
quantized_by: zrlu
SuperQwen3.8-27b-abliterated-GPTQ-Int4-sym-G128-MTP-BF16 (B70)
GPTQ-INT4 quantization, purpose-built for single Intel Arc Pro B70 (Xe2)
inference via vLLM XPU (Windows Docker Desktop + WSL2 or native Linux), and for
pi/opencode agent workloads.
- Model name: SuperQwen3.8-27b-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70
- Original model: Jiunsong/SuperQwen3.8-27b-abliterated
(BF16 "supertune" full-precision release) - Type: quantized derivative checkpoint (GPTQ-INT4), MTP (spec-decode) heads
preserved in BF16 — required by the vLLM B70 MTP4 recipe.
License
Apache-2.0 — inherited unchanged from the base model (Qwen3.8-27B lineage).
Weight-only quantization; no rights are widened.
Quantization (self-hosted, reproducible)
- Tool:
gptqmodel==7.3.2on CUDA (RTX 5090), lazy checkpoint loading;
calibration wikitext-2 (or bundled fallback text). - Contract:
bits=4, group_size=128, desc_act=false, sym=true, lm_head=false, dynamic={"-:.*mtp.*":{}}(MTP tensors stay BF16). - Source BF16 (51.8 GB) → quantized ~18.2 GB / 5 shards; config arch
Qwen3_5ForConditionalGeneration,image_token_id 248056.
Serving (Arc Pro B70)
Served with the B70 one-click image (zrlu/qwen38-27b-arc-pro-b70:2026.08.24,
vLLM XPU 0.27.2rc1.dev77 + kernels 0.1.12.3, MTP4 + draft-INT4 S+M1 overlay,
prefix caching, qwen3_xml tool parser, fp8 KV):
docker run -d --name qwen38-super --init --device /dev/dxg --ipc=host --shm-size=16g \
--restart unless-stopped --security-opt seccomp=unconfined --cap-add SYS_NICE \
-p 127.0.0.1:8000:8000 \
-v /usr/lib/wsl/lib:/usr/lib/wsl/lib:ro -v /usr/lib/wsl/drivers:/usr/lib/wsl/drivers:ro \
-e HF_REPO="zrlu/SuperQwen3.8-27b-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70" \
zrlu/qwen38-27b-arc-pro-b70:2026.08.24
First start auto-downloads this repo into /model (~18 GB). Full setup,
pi-agent wiring and benchmark methodology live in the
GitHub repository
(benchmarks included).
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("zrlu/SuperQwen3.8-27b-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70")
m = AutoModelForCausalLM.from_pretrained(
"zrlu/SuperQwen3.8-27b-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70",
device_map="auto")
Signed: zrlu. Quantization GPU: RTX 5090; GPU of use: Intel Arc Pro B70.