← back to catalog · registered 2026-11-20 00:04

xenoautomaton/Qwen3.8-27B-abliterated-cyber-preview-W8A16

xenoautomaton 27B second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/xenoautomaton%2FQwen3.8-27B-abliterated-cyber-preview-W8A16"
Response includes
  • classification m1
  • files 14
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M1
Primary method

Direct removal

No other method signals detected in this model.
Confidence
MEDIUM
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=0 (base model)
  • no specific method indicators - defaulting to M1 (most common)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
MEDIUM
Why we say so
primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)
Downloads · 30-day
44
Likes
0
Model age
2d ago
created 2026-11-19
Downloads over time
Now 44 → from 0 ↑ 0%
0163248 0 on Nov 20 44 on Nov 22 Nov
Nov 20 → Nov 22 · 3 snapshots · spans 2 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Tags
transformers safetensors qwen3_5 image-text-to-text compressed-tensors int8 vllm text-generation conversational base_model:hotdogs/Qwen3.8-27B-abliterated-cyber-preview base_model:quantized:hotdogs/Qwen3.8-27B-abliterated-cyber-preview license:apache-2.0

Related

Total size
29.4 GB
Files
14
Quantizations
1
Registered
2026-11-20 00:04
Last updated on HF
2026-11-19 23:33

Files by quantization

Auxiliary files 14 files 29.5 GB
model-00001-of-00002.safetensors 25.9 GB 4f2b5d08 download
model-00002-of-00002.safetensors 3.53 GB dce62fcd download
tokenizer.json 19.1 MB 87a7830d download
model.safetensors.index.json 194 KB 05eec066 download
LICENSE 11.1 KB d6456956 download
chat_template.jinja 8.74 KB c0c686f9 download
tokenizer_config.json 6.99 KB 773a6327 download
config.json 4.57 KB 3c1308af download
README.md 3.84 KB f815a0d9 download
.gitattributes 1.53 KB 52373fe2 download
quantization-provenance.json 1.33 KB 102ffda8 download
processor_config.json 1.16 KB 33818c7f download
generation_config.json 213 B 79c8cce3 download
SHA256SUMS 198 B 37e3c7e9 download

README current version from Hugging Face


library_name: transformers
license: apache-2.0
base_model: hotdogs/Qwen3.8-27B-abliterated-cyber-preview
base_model_relation: quantized
pipeline_tag: text-generation
tags: [compressed-tensors, int8, vllm, qwen3_5]

Qwen3.8-27B Cyber Preview — W8A16

A self-produced INT8 weight-only compressed-tensors checkpoint, distinct
from Q6_K/Q4_K_M GGUF. Weights total about 31.62 GB.
Source revision: 2fbed4d2a8cbf450797dfdcd65bb5ccef74569df.

Created with llmcompressor 0.15.0 and compressed-tensors 0.20.1 using data-free
RTN, symmetric groups of 128, and 16-bit activations. Vision weights, embeddings,
lm_head, MTP tensors and small GDN control projections were kept unquantized.
See quantization-provenance.json for the complete exclusions and verified source.

Tested vLLM deployment

vLLM 0.34.0 / PyTorch 2.14.0+cu130, BF16 activations, A100 40GB. This native
checkpoint does not require the experimental GGUF plugin. Both ordinary serving
and MTP=1 passed the functional checks; MTP=1 was selected for the deployed API.
Observed GPU memory was about 34.4 GiB at the tested 32K context.

vllm serve xenoautomaton/Qwen3.8-27B-abliterated-cyber-preview-W8A16 \
  --host 127.0.0.1 --dtype bfloat16 --max-model-len 32768 --max-num-seqs 1 \
  --max-num-batched-tokens 4096 --gpu-memory-utilization 0.90 \
  --language-model-only --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Private downloads require your own HF authentication. Use a separate inference
API key when exposing the service. The deployed public service also requires
that key on non-OpenAI inference/control routes.

Variant / engine Prompt tokens Output tokens End-to-end tok/s Observed stream tok/s
W8A16 / vLLM 0.34.0 MTP=0 4832 256 39.8 41.5
W8A16 / vLLM 0.34.0 MTP=0 32 256 41.0 41.9
W8A16-MTP / vLLM 0.34.0 MTP=1 4832 256 56.4 60.0
W8A16-MTP / vLLM 0.34.0 MTP=1 32 256 66.0 68.8

Validation scope

Measured on one NVIDIA A100-SXM4-40GB with a 32,768-token serving context,
one sequence, temperature 0 and thinking disabled per request. Tables show
medians excluding retained warmups. Stream rates are observed from SSE events,
not isolated GPU kernel measurements. Cached and cold TTFT differ; raw records
retain that distinction. Six functional checks cover arithmetic, sorting, Chinese
translation, executing a small generated Python function, tool-call generation
and retrieval. This is not a comprehensive accuracy benchmark. Quantization is
lossy. No RL or further fine-tuning was performed. Vision inference was not tested.

The source's native context remains 262,144 tokens. The 32,768-token runtime
setting does not extend RoPE or imply that the entire native context was tested
on this 40GB card. All upstream credits and Apache-2.0 terms remain applicable.

One-cell Colab recovery

Download restore_colab.ipynb and open it in an A100 40GB Colab runtime. Run the single code cell to restore the pinned runtime, download these published weights, and start authenticated vLLM + MTP through Cloudflare Tunnel. No requantization is needed. See recovery details.

The current launcher automatically fits the context to GPU memory at 95% utilization. On the tested A100 40GB it fits 99008 total tokens; an actual 98376-token retrieval probe passed. The native 262144 setting failed the KV-memory check. Earlier 32K performance measurements above remain historical. Enlarging the limit retained about 69 tokens/s on the short-input benchmark and 60 tokens/s on the 4.8K-input benchmark. These are limited functional/performance tests, not comprehensive quality evaluation.

README history 4 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-19 Document public Colab recovery notebook and tested context limit 7114d76 3.8 KB
    Loading...
  2. 2026-11-19 Document verified A100 runtimes, MTP performance and validation limits 16298bd 3 KB
    Loading...
  3. 2026-11-19 Document verified A100 runtimes, MTP performance and validation limits ed6d196 3 KB
    Loading...
  4. 2026-11-19 Add self-quantized W8A16 checkpoint and A100 inference validation f4acfd3 2 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration