license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
language:

• en
• zh
  tags:
• qwen3
• qwen3.8
• uncensored
• abliterated
• orcarouter
• gsq
• rco
• gguf
• quantized
• speculative-decoding
• mtp
• text-generation
• long-context
• multimodal
• vision
• tool-use
• 16gb-vram
  model-index:
• name: Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-Uncensored
  results:
  • task:
    type: text-generation
    dataset:
    name: needle-retrieval-15k
    type: custom
    metrics:
    • name: exact-match
      type: exact_match
      value: 6/6
  • task:
    type: text-generation
    dataset:
    name: toolcall-json-full-14-case
    type: custom
    metrics:
    • name: pass-rate
      type: pass_rate
      value: 9/14
  • task:
    type: text-generation
    dataset:
    name: gpqa-diamond
    type: gpqa_diamond
    metrics:
    • name: accuracy
      type: acc
      value: 75.25
  • task:
    type: text-generation
    dataset:
    name: wikitext-2
    type: wikitext2
    metrics:
    • name: perplexity
      type: perplexity
      value: 6.17
  • task:
    type: text-generation
    dataset:
    name: ifeval
    type: ifeval
    metrics:
    • name: prompt-strict-accuracy
      type: acc
      value: 70.24
    • name: instruction-strict-accuracy
      type: acc
      value: 76.26
  • task:
    type: text-generation
    dataset:
    name: truthfulqa
    type: truthful_qa
    metrics:
    • name: mc1
      type: acc
      value: 77.60
    • name: mc2
      type: acc
      value: 81.98
  • task:
    type: text-generation
    dataset:
    name: xstest-safe
    type: custom
    metrics:
    • name: over-refusal-rate
      type: rate
      value: 0.0

Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-Uncensored

A 3-bit GGUF quant of the uncensored OrcaRouter Qwen3.8-27B, carrying an S1-trained MTP draft head and a custom OrcaRouter-native imatrix. Built to run on a single 16 GB GPU.

A 9.75 GiB, mixed-precision GSQ/RCO quant of orcarouter/Qwen3.8-27B-Uncensored, preserving the MTP head and validated for 16 GB GPUs.

This is a quantization of orcarouter/Qwen3.8-27B-Uncensored, not a new fine-tune. The underlying model is Qwen3.8-27B; the surgical uncensoring comes from OrcaRouter; the quantization work here follows the GSQ/RCO methodology developed by ISTA-DASLab. The checkpoint is designed to substantially reduce learned refusal behavior. This is not a guarantee of universal compliance — the deployer is responsible for compliant use.

Quick specs

What I changed

Starting from the OrcaRouter uncensored checkpoint, I produced this GGUF using a GSQ/RCO-based non-uniform allocation and IQ3_XXS packing, then trained the MTP draft head (S1) and built a custom OrcaRouter-native importance matrix.

The important honesty clause: I reproduced ISTA-DASLab's published per-tensor RCO allocation verbatim from their stock-base artifacts. I did not independently re-run the multi-GPU budget search — same map, applied to the uncensored base.

The imatrix is custom and OrcaRouter-native; it differs from ISTA's on the edit-mask tensors. ISTA's published imatrix was also validated against this file. If a change makes the benchmark prettier but the model worse to actually use, it does not ship.

Tested performance (RTX 5070 Ti 16 GB, build2 llama.cpp, this exact file)

MTP head (S1-trained)

The offline k1 gain is large and generalizes (measured on a distribution never trained on). Serve-time throughput at n-max 2 is bounded by verification cost, so the head's drafting advantage shows as a modest serve-time gain — not a one-to-one map of the offline gain. Needle 6/6 re-verified on this file.

Usage

Froggeric template is required for the intended tool/reasoning behavior.

llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --alias Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-Uncensored \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2

-b 2048 is required for depth-0.5 retrieval fidelity. This is a documented chunking artifact: a smaller batch splits the retrieval span and the depth-0.5 needles are lost. Keep -b 2048 -ub 2048 for retrieval-sensitive workloads.

Vision: projector files live in this repo under mmproj/ (BF16 or Q8_0). Start the server with one, then send a multimodal chat request (skip both flags for text-only):

llama-server -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf [same flags as above]
# then POST /v1/chat/completions with content blocks including {"type":"image_url",...}

Ollama (text-only; tool/reasoning behavior is uncertified — use llama-server + Froggeric for the certified path):

FROM ./Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf
PARAMETER num_ctx 32768
PARAMETER num_gpu 99
PARAMETER temperature 0.6

• Dual-engine routing: for ≤128K context use standard build2 (fast path; measured ~82 vs ~72 t/s at short context). For >128K up to 256K use the BrunoPPassini KV-ring fork — it holds 200K context and passes 2/2 needle retrieval where build2 cannot fit the KV cache.

Honest limitations

Aggressive 3-bit quant — do not expect BF16 behavior. Knowledge recall is weaker than higher-bit variants; the model leans on retrieval/tools instead. Vision + long context is tight on 16 GB (split text/vision profiles or offload the projector).

1. RCO allocation is reproduced verbatim from ISTA-DASLab's stock-base artifacts (not re-searched) and applied to the uncensored base.
2. The imatrix is custom OrcaRouter-native; it differs on the edit-mask tensors. ISTA's published imatrix was also validated against this file.
3. Abliteration Tax — GPQA-Diamond 75.25% (harness-conditional). Measured with thinking ON, a generous thinking budget, and temp 0 over 198 questions; not a matched comparison to ISTA's stock-aligned 88.89%. The ~13-point gap is the documented cost of uncensoring — surgical rotation of the residual stream combined with 3-bit noise over long reasoning chains. It is a limitation of the abliterated base, not of the GSQ-RCO map, which is proven clean by WikiText-2 perplexity 6.17.
4. The v1.1 Huihui line is frozen/deprecated in favor of this OrcaRouter line. v1.1 is retained for reproducibility only.
5. Quant layout: IQ3_XXS trunk (~3.06 bpw), MTP head Q6_K, F32 norms preserved.

MTP output may differ from serial generation on quantized targets; fix the seed and runtime config for reproducibility, and disable spec when exact serial behavior matters.

Files in this repo

GGUF embeds tokenizer metadata — no separate tokenizer needed for llama.cpp/Ollama/LM Studio/Pi. The vision projector is included under mmproj/ (BF16 default, Q8_0 alternative); it is optional and text-only use works without it.

Credits

• Qwen — architecture + pretrained weights: Qwen3.8 repo, Qwen3.8-27B
• OrcaRouter — uncensored checkpoint: orcarouter/Qwen3.8-27B-Uncensored
• GSQ — Dadgarnia, Tabesh, Nikdan, Helcig, Kurtic, Kleinegger, Alistarh (2026): paper, code
• RCO — Helcig & Alistarh (2026): paper, code
• ISTA-DASLab quant release — Qwen3.8-27B-GSQ-RCO-GGUF (allocation source; imatrix also validated)
• Froggeric — Qwen-Fixed-Chat-Templates
• llama.cpp/GGML — runtime + format; IQ3_XXS is their standard type, nothing custom here

Reproducibility

• Base: orcarouter/Qwen3.8-27B-Uncensored
• Allocation: REF-IQ3_XXS-mtp.rco-allocation.txt (in this repo) — ISTA-DASLab's published per-tensor RCO map, reproduced verbatim, applied via llama-quantize --tensor-type-file
• Imatrix: imatrix.dat (in this repo) — custom OrcaRouter-native; ISTA's published matrix also validated
• MTP head: S1-trained draft head (Q6_K), F32 norms preserved
• Runtime: den_llama.cpp (build2)
• GPU: RTX 5070 Ti 16 GB, Windows 11

Versions

Changelog

• v2.0 (current): OrcaRouter uncensored base; S1-trained MTP head; custom OrcaRouter-native imatrix; ISTA-verbatim RCO allocation. Needle 6/6, GPQA-Diamond 75.25% (harness-conditional), WikiText-2 ppl 6.17, toolcall 9/14, IFEval 70.24/76.26, TruthfulQA 77.60/81.98, 0% over-refusal. MTP offline k1 +0.278 over native; serve +13.6% t/s / +12.4% accept vs ISTA-imatrix build.
• v1.x (Huihui line) — frozen/deprecated. Superseded on every axis by this OrcaRouter line, except the native-head toolcall 10/14 vs 9/14 on this file. Retained for reproducibility only.

Roadmap

• Bilingual (EN/ZH) ModelScope card: deferred to upload time if requested. This staging card is English-only.

Community quant, not affiliated with Qwen, OrcaRouter, Huihui, or ISTA-DASLab. Bug reports: include model revision, llama.cpp build, ctx/KV/spec settings, sampler. "Feels worse" is fine; reproducible info makes it actionable.