license: apache-2.0
base_model: Jackrong/Qwopus3.8-27B-Flash
tags:

• abliterated
• uncensored
• awq
• vllm
• qwen3.5
  language:
• en
• ja
  source_path: /home/nagi/mochievi/out/hf_modelcard_qwopus38_awq_draft.md
  sha256:
  date:
  model:
  host:

Status: DRAFT / evaluation in progress. Refusal-rate and tool-call parser results below are preliminary and expected to change as testing continues.

Qwopus3.8-27B-Flash-Heretic-AWQ

A 4-bit AWQ (compressed-tensors) quant of a heretic-abliterated (uncensored) version of Jackrong/Qwopus3.8-27B-Flash, which is itself an agent-trajectory (Claude/GPT) Flash-distillation of Qwen/Qwen3.8-27B (Qwen3.5 hybrid architecture: Gated DeltaNet linear-attention layers + standard attention + MoE).

This repo does not perform the abliteration — it AWQ-quantizes the already-abliterated fp16 checkpoint (selected below) for TP=2 vLLM serving. Read "Known issues" before using this for anything that depends on tool calling or reliable refusal behavior — both are weaker than a finished product.

What this is

1. Base: Jackrong/Qwopus3.8-27B-Flash (Apache 2.0, upstream describes itself as "an independent experimental release for research, local evaluation, and technical exploration... has not undergone broad safety evaluation").
2. Abliteration: run with heretic v2.0.0.dev0, 6 Optuna trials, best trial (trial 4) auto-selected by heretic's Pareto sort on the mlabonne/harmful_behaviors refusal/KL objective. Baseline refusals 98/100 (KL 0 by definition) → trial 4 refusals 84/100, KL divergence 0.011. Only 6 of heretic's default 200 trials were run (compute-budget-limited), so this is a shallow search of the direction/parameter space, not an exhaustively tuned abliteration — see "Known issues."
3. Quantization: the fp16 merged export was converted to 4-bit AWQ via compressed-tensors. The raw AWQ export had a model.language_model. key prefix mismatch against vLLM's expected module names (MergedColumnParallelLinear had no .data attribute on load) and a stale model.language_model. prefix left inside config.json's quantization_config.ignore list (337 entries) that a naive key-fix script does not rewrite. Both were fixed by stripping the prefix from the safetensors index/state-dict keys and hand-patching the ignore list with the same rule before the checkpoint would load in vLLM.

How to run

Tested working command (vLLM 0.27.1, 2x GPU, compressed-tensors AWQ):

env PATH="<venv>/bin:$PATH" \
  <venv>/bin/vllm serve /path/to/qwopus38-27b-heretic-awq-fixed \
  --tensor-parallel-size 2 \
  --max-model-len 131072 \
  --quantization compressed-tensors \
  --gpu-memory-utilization 0.90 \
  --max-num-seqs 64 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --served-model-name qwopus27b-heretic-awq

Notes on the flags, from what actually broke during bring-up:

• --gpu-memory-utilization 0.90: at 0.82 the KV cache initializer refused to start (4.09 GiB KV cache is needed, which is larger than the available KV cache memory (3.59 GiB)). Raise this before anything else if you hit that error.
• --max-num-seqs 64: the default (256) exceeds this architecture's Mamba cache block budget (max_num_seqs (256) exceeds available Mamba cache blocks (228) — the Gated DeltaNet linear-attention layers use a Mamba-style cache, not plain KV). Any value comfortably under the block limit works; 64 was tested.
• --max-model-len 131072: the model's declared max (262144) does not fit the KV budget on a 2-GPU load at these settings; 131072 loads and runs.
• env PATH="<venv>/bin:$PATH": vLLM shells out to a bare ninja for JIT compilation. If your venv's bin/ isn't on $PATH, the worker dies with RuntimeError: [Errno 2] No such file or directory: 'ninja' even when ninja is pip-installed in that venv.

Measured numbers (2x GPU, TP=2, this AWQ build — all figures are single-run spot checks, not averaged benchmarks)

Speed (/v1/completions, max_tokens=200, temperature=0):

Refusal rate (mlabonne/harmful_behaviors test[:100], heretic's own keyword-based scorer re-implemented against /v1/chat/completions, max_tokens=100 greedy — same method and token budget as the heretic report, but through vLLM's chat endpoint rather than heretic's own HF-transformers harness):

• This AWQ build: 42/100 refused (vs. 84/100 for the fp16 trial-4 checkpoint in the heretic report — the gap is not fully explained; candidates include the quantization itself, the 100-token cap truncating <think> blocks before an answer is reached, and running through vLLM's chat/tool-call path instead of heretic's own eval harness).
• For reference, the production dense-AWQ model this bench host normally serves scored 13/100 under the identical harness — this build is less compliant (more refusals) than that baseline, despite being the "uncensored" one. Note the keyword scorer has known false-positive refusals on both models (e.g. flags articles that merely use words like "traffic" or "violat-").

Output length / language behavior (20 prompts, mixed everyday/business/technical/terse, temperature 0.7, max_tokens=2048):

• No runaway generation or repetition loop observed in either thinking-on or thinking-off mode; nothing hit the 2048-token cap.
• Mean output: 518.6 tok (thinking ON) / 547.5 tok (thinking OFF) — shorter than the reference dense model's 947 tok mean on the same prompts. No "always writes an essay" habit found in this sample.
• Terse-instruction prompts ("answer in one word") are followed correctly with thinking OFF (2-token answers). With thinking ON, the same prompts still answer correctly but burn 39-75 reasoning tokens first.
• Japanese naturalness (5 manually rated samples, thinking OFF): mean 4.2/5, no worse than the reference model.
• Language drift: with thinking ON, the <think> reasoning block frequently switches to English even when the prompt and the final answer are in Japanese (4 of 5 spot-checked samples). Thinking OFF stayed in Japanese throughout. If you care about a consistent Japanese chain-of-thought, run with chat_template_kwargs: {"enable_thinking": false}.

Known issues (do not treat as production-ready)

• Tool calling does not parse. The model does emit the intended <tool_call>{...}</tool_call> content when asked to call a function, but --tool-call-parser qwen3_coder does not extract it into a structured tool_calls array — it stays as raw text inside content. Root cause not diagnosed (likely a chat-template/parser format mismatch specific to this checkpoint). If your use case depends on OpenAI-style structured tool calls, this build is not usable as-is.
• Abliteration is shallow. Only 6 of heretic's default 200 trials were run, and the refusal-rate improvement (98→84/100 on the fp16 checkpoint, before quantization) is modest compared to other public heretic-abliterated Qwen releases. Refusal behavior after AWQ quantization measured even more inconsistent (42/100 on the harness described above) — quantization may have interacted with the abliteration in ways that were not investigated further.
• Not an official/first-party release. This is a from-source AWQ conversion of a third-party experimental fine-tune, produced for local infra testing. It has not gone through any safety review beyond the keyword-based refusal count above.

Uncensored model disclaimer

This is an abliterated model: its built-in refusal behavior has been intentionally weakened relative to the base model. It will comply with a wider range of requests, including ones a stock instruction-tuned model would decline, and — per the measurements above — its refusal rate is inconsistent and not reliably lower than the base model's for every category of prompt. There is no moderation layer on top of this checkpoint. Use is entirely at your own judgment and responsibility; you are responsible for how you deploy and use its outputs.

Credits

• Base architecture: Qwen team / Alibaba (Qwen/Qwen3.8-27B).
• Flash agent-trajectory distillation: Jackrong/Qwopus3.8-27B-Flash (Apache 2.0).
• Abliteration method: heretic (p-e-w).
• This repo: heretic run (trial selection) + AWQ quantization + vLLM key-compatibility fix, hosted as-is.

License follows upstream: Apache 2.0 (inherited from Jackrong/Qwopus3.8-27B-Flash, itself inheriting from Qwen/Qwen3.8-27B).

日本語まとめ

ステータス: DRAFT / 評価継続中。 拒否率・tool-callパーサの結果は暫定値で今後更新される見込み。

Jackrong/Qwopus3.8-27B-Flash(Qwen3.8-27BをClaude/GPTエージェント軌跡でFlash蒸留したモデル、Apache 2.0)をheretic v2.0.0.dev0で無検閲化(6trial中trial4を採用、拒否率98→84/100、KL 0.011)し、そのfp16マージ済みチェックポイントをAWQ 4bit(compressed-tensors)へ量子化したもの。AWQ変換時にmodel.language_model.プレフィックスのズレでvLLMがロードできない不具合があり、safetensorsキーとconfig.jsonのignoreリスト双方を同じルールで手動修正して解消した。

既知の問題: (1) tool_call出力はcontentに文字列として残るだけでqwen3_coderパーサが構造化できない(未解決)。(2) 拒否率はAWQ変換後の計測で42/100と、fp16トライアルの84/100より変動が大きく安定していない。(3) heretic探索は既定200trial中6trialのみで打ち切っており、探索不足の可能性が高い。(4) thinking ON時、日本語プロンプトでも推論部が英語になることがある(thinking OFFなら問題なし)。速度はC1 45.3 tok/s、C4合計130.9 tok/s(2GPU TP=2)。無検閲モデルにつき、拒否率の低さは保証されず、利用は自己責任。