license: apache-2.0
base_model: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
base_model_relation: quantized
library_name: gguf
tags:

• gguf
• llama.cpp
• gsq
• rco
• iq3_xxs
• quantized
• abliterated
• uncensored
• 27b
• qwen3.8

Qwen3.8-27B Blackfrost-Abliterated · GSQ-RCO-IQ3_XXS GGUF (~9.8 GiB)

TL;DR: First Blackfrost-lineage GSQ-RCO allocation quant. ISTA's published RCO allocation table (866 tensors) is applied 1:1 to the official Blackfrost abliterated BF16 weights — no grid training needed. Result: at 9.8 GiB it matches the 15.7 GiB daily-driver quant on behavioral batteries and is the fastest quant in our lineup on real-prompt replay (RTX 5090, llama.cpp b10930). Sized for the RTX 5090's 32 GB: weights + 256K context stay fully resident on one GPU, zero offloading.

中文摘要：全网首个 Blackfrost 血统的 GSQ-RCO 分配表量化档。把 ISTA 官方公布的 RCO 分配表（866 张量）直接打到官方 Blackfrost 去审查 BF16 权重上，无需任何网格训练。9.81 GiB 体积下行为级电池与本机 15.7G 主档同分，真实负载回放全场最快（138.9 tok/s @ RTX 5090），并为 5090 的 32G 显存量身定做——权重 + 256K 上下文单卡全驻留，零卸载。所有数字均为本机实测，同 harness 同基线。

Why this exists

GSQ-RCO (ISTA / IST-DASLab) learns a per-tensor grid-format allocation under a bit budget. The allocation transfers to abliterated copies of the same architecture (same tensor shapes, same budget), so the learned table can be applied to the official Blackfrost abliterated weights with a single llama-quantize pass. Method credit goes to RentedNoodle, who first demonstrated this transfer on a Huihui-base abliterated model; this repo is the same allocation applied to the cleaner single-pass Blackfrost lineage — and the lineage difference turns out to be measurable (see benchmarks).

Built for the RTX 5090 (32 GB) — measured, not marketed

• One GPU, no offloading: 9.81 GiB of weights leaves ~22 GB of the 5090's 32 GB for KV cache — we run 256K-context sessions fully resident (202K deep-needle measured on this exact setup).
• Fastest quant we've ever measured on replay: 138.9 tok/s decode on 30 real user prompts, +19% over our 15.7 GiB mixed 4/5-bit daily driver — at one-sixth-less-the-bytes and equal HardBattery score. Decode is weight-bandwidth-bound, so the small file is the speed.
• The whole point of the size: this quant is the sub-10-GiB tier of a project that chased daily-driver behavioral quality (HardBattery 14/17, same as the 15.7 GiB main) at a size that leaves the 32 GB card headroom for long context. On smaller cards it also leaves room for big KV.
• Runs anywhere, not Blackwell-locked: the tensors are IQ/K-format (IQ3_XXS/IQ3_S/IQ2_S mix with Q4_K/Q6_K anchors) — any GPU that runs llama.cpp runs this. The RTX 5090 numbers above are simply where we measured. (Explicit note: this quant contains no NVFP4 tensors — NVFP4 is a different, Blackwell-native quant family.)

Method (reproduce in one command)

Inputs: official Blackfrost abliterated BF16 GGUF + ISTA official imatrix + the allocation table shipped in this repo (gsq-rco-iq3xxs-allocation-table.txt, one tensor_name=ggml_type per line).

llama-quantize \
  --imatrix <ista-official.imatrix> \
  --tensor-type-file gsq-rco-iq3xxs-allocation-table.txt \
  <blackfrost-abliterated-bf16.gguf> \
  <output.gguf>

Benchmarks

All numbers measured locally on RTX 5090 32GB / llama.cpp b10930, same harness, same golden baseline, temp 0. Nothing is quoted from other people's cards.

Protocol notes:

• KLD is against saved BF16 golden logits (f16 KV, identical corpus/chunking for all quants). Treat it as a weight-fidelity indicator, not a quality ranking — within the 0.02–0.08 band our behavioral tests never separated quants; 0.1137 is outside that band and does reflect a real 3-bit cost.
• Easy battery: knowledge quiz + open-ended generation (auto-judged) + IFEval-style instruction following. HardBattery: math with brute-force verified ground truth + sandbox-executed coding + hard logic — the only battery that separated model generations in our setup.
• 202K needle: deep-context retrieval needle at 202K context.
• Replay: 30 real user prompts, paired across quants, temp 0 / 400 output tokens, with the same BF16 speculative draft model (decode tok/s and draft acceptance rate). This is the only speed comparison we trust; synthetic benchmarks lied to us before (+11% real-world delta measured as "4% noise" synthetically).
• vs the same-allocation RentedNoodle card: 69% lower KLD, better open-ended generation, +12% needle accuracy, higher draft acceptance, same speed — single-pass clean abliteration quantizes measurably better than multi-layer crude abliteration.

Known limits

• 3-bit fidelity cost is real: KLD 0.1137 vs 0.0387 for our 14.95G mixed quant. Pick by VRAM budget.
• Long-context (202K) decode is still behind the bigger quants (63.2 vs 70.1 tok/s).
• A 450-case refusal-bench protocol has not yet been run on this exact quant+engine combo (it differs across stacks by up to 3×, so we won't quote weight-level numbers).

Usage

Standard GGUF — any recent llama.cpp build:

llama-server -m <this-model>.gguf --jinja -ngl 999 -c 65536

The chat template keeps the chain-of-thought channel, so keep API max_tokens ≥ 600.

Files

Credits & license

• Base weights: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
• Method: IST-DASLab GSQ + RCO (allocation table & imatrix); RentedNoodle for the allocation-transfer recipe
• License inherits the base model (Apache-2.0 per Qwen lineage) — check the base repo for the authoritative terms

Disclaimer

This is an abliterated (refusal-reduced) model intended for local research and personal use. It will comply with harmful requests more readily than the base alignment intends. You are responsible for what you do with it.