license: apache-2.0
base_model: Qwen/Qwen3.8-27B
base_model_relation: finetune
library_name: transformers
pipeline_tag: image-text-to-text
tags:

• qwen
• lora
• reasoning
• opus-distill
• vision
• mtp
• causal-lm
• v2

Derivative checkpoint notice

This repository is an architecture-aware refusal-direction ablation of
barozp/Qwen3.8-27B-Opus-Distill-v2 at pinned revision
2c1dcd2dd468b67a893fffbe961cbbc1781465b4.

• Method: rank-1 refusal-direction weight orthogonalization
• Residual position: 46 (forced)
• Lambda: 1.4
• Edited tensors: 131
• Preserved tensor keys: 1199
• Unexpected tensor changes: 0
• Structural verification: PASS
• Tool: nanofatdog/LLM-abliterate@f01cec9633a591657eea5934d1ce4a5999800844
• Built: 2026-08-29T15:14:02.807023+00:00

The edited set contains all 64 MLP residual writers, 48 linear-attention output
projections, 16 full-attention output projections, both MTP residual writers,
and the language embedding matrix. Vision tensors and lm_head are preserved
unchanged. The complete base tensor key set, including the vision/MTP sidecar,
was retained.

Selected sweep row:

{
  "lam": 1.4,
  "refusals": "2/32",
  "refusal_pct": 6.25,
  "capability": "35/40",
  "capability_pct": 87.5
}

Direction metadata:

{
  "position": 46,
  "mode": "nothinking",
  "d": 12.894421577453613,
  "auc": 1.0,
  "hidden": 5120,
  "n_good": 32,
  "n_bad": 32
}

Refusal suppression does not make generations accurate, benign, or suitable
for every deployment. Re-evaluate capability, factuality, and behavior on the
actual prompts and inference settings you intend to use.

Qwen3.8-27B-Opus-Distill-v2

Qwen/Qwen3.8-27B fine-tuned (LoRA, merged) on verified-genuine Claude Opus
reasoning traces. This release fixes a reasoning-loop bug reported against
the first release (barozp/Qwen3.8-27B-Opus-Distill,
"v1") while keeping the reasoning-quality gains v1 was known for.

Why this release exists

A community member (zxbc2023) reported a
clean, deterministic bug on v1: certain stacked output-format constraints
(most notably "no prose" combined with a second constraint like "no markdown") sent the model into a non-converging self-verification loop —
burning the entire token budget on repeated "wait, let me re-check…" passes
with zero visible output. Their repro was exact and reproducible at
temp=0, and they correctly diagnosed it as likely inherited from the
distillation data rather than a capability gap. Full credit to them for the
report that made this fix possible.

Root cause. Investigating v1's training data (opus-reasoning-distill-train)
showed that 83% of it had actually been sourced from Jackrong's
TraceInversion datasets — reasoning traces reconstructed by a small
auxiliary model (Trace-Inverter-4B) from a compressed summary, not the
teacher model's real output (openly documented on those datasets' own
cards). That reconstruction process appears to have baked in a rigid
"enumerate constraints, then re-verify" template — plausibly both the
source of v1's reasoning-quality gains and the loop bug, two sides of the
same mechanism.

The fix, in two steps:

1. First attempt: retrain purely on a genuine-trace dataset (lordx64/reasoning-distill-claude-opus-4-7-max).
   This fixed the loop bug completely but collapsed the reasoning-transfer
   gain (GPQA-Diamond dropped back to base/random level) — the domain mix
   mattered as much as trace authenticity.
2. This release: reconstructed v1's exact prompt distribution, but
   replaced every row traced back to Jackrong's fabricated sets with the
   verified genuine trace for the same prompt (from lordx64 or
   Roman1111111/claude-opus-4.6-10000x). Then ran a full-dataset scan
   (30 parallel LLM-judge passes over all 14,250 rows) for the specific
   pathological pattern and removed every flagged row (169) plus every row
   whose source was never verified against a known-genuine dataset (2,429).
   Net: 14,250 → 11,716 rows, 100% traced to verified sources.

Bug fix validation

Re-ran zxbc2023's exact repro (temp=0, seed=1234, fib(n) with stacked
constraints) against this model:

All three converge quickly with </think> closing normally — no loop, no
budget exhaustion.

Benchmark results

Measured with lm-evaluation-harness, 0-shot, loglikelihood (multiple-choice),
chat template OFF, QUICK mode (--limit 500) — identical protocol to v1,
so the Δ column is the meaningful signal.

Reading the table: knowledge (MMLU) and language modeling (wikitext) are
unaffected; reasoning (ARC, GPQA) shows a large, real gain — slightly
smaller than v1's (expected: v1's gain was partly driven by the same
fabricated content that caused the bug), but still far above noise. GPQA in
particular is a same-protocol delta vs. base, not comparable to Qwen's own
published 89.2 (thinking-mode-on, different harness) — see v1's card for
the full caveat.

Model overview

• Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
• Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
• LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers.
• Training data: barozp/opus-reasoning-distill-v2 (11,716 examples, 100% verified-genuine Opus traces).
• Training run: 1 epoch (696 steps), lr=1e-4 cosine with 3% warmup, effective batch 16, MAX_SEQ=4096, bf16, ~4h15m on an A100 80GB.
• Final validation loss: 0.4334 — lower than both v1 (0.4647) and the intermediate pure-genuine attempt (0.805), despite less data.
• Vision + MTP: carried over from base, never trained.

Quick start

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill-v2", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill-v2")

Text-only inference also works via AutoModelForCausalLM.

Related

• Qwen3.8-27B — the base model.
• barozp/Qwen3.8-27B-Opus-Distill — v1. Still works well for most use cases; this release specifically fixes the stacked-constraint reasoning-loop edge case.
• barozp/opus-reasoning-distill-v2 — the training dataset, with full provenance notes.
• Thanks to zxbc2023 for the bug report that led to this release.

Quantized releases of this checkpoint:

• barozp/Qwen3.8-27B-Opus-Distill-v2-FP8 -- block-wise FP8 (e4m3, 128x128) for CUDA serving stacks; paired BF16-vs-FP8 evals show deltas within noise.
• barozp/Qwen3.8-27B-Opus-Distill-v2-GGUF -- llama.cpp / Ollama / LM Studio quants (BF16 down to IQ1_M), mmproj vision, MTP self-speculative decoding.
• barozp/Qwen3.8-27B-Opus-Distill-v2-MLX-4bit and MLX-8bit -- Apple Silicon mlx-vlm conversions.