license: apache-2.0
base_model: huihui-ai/Huihui-Qwopus3.5-27B-v3-abliterated
tags:

• qwen3.5
• nvfp4
• quantized
• abliterated
• opus-distilled
• vllm
• compressed-tensors
• blackwell
  library_name: transformers
  pipeline_tag: image-text-to-text
  model_type: qwen3_5
  quantized_by: Lna-Lab

Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4

NVFP4 quantized version of huihui-ai/Huihui-Qwopus3.5-27B-v3-abliterated — an abliterated Qwen 3.5 27B distilled from Claude Opus reasoning patterns (Jackrong/Qwopus3.5-27B-v3).

~51 GB → 19.8 GB. Single NVIDIA Blackwell GPU.

What Makes This Different

Qwopus = Qwen 3.5 + Opus-style reasoning. The base model was trained to produce stable, structured reasoning even at short output lengths — where vanilla Qwen3.5 needs more tokens to reach peak quality.

This makes Qwopus especially suited for agentic workflows where each tool call or code snippet needs to be precise in a short response.

Key Specs

Quickstart

vLLM

vllm serve Lna-Lab/Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4 \
    --max-model-len 32768 \
    --reasoning-parser qwen3

With tool calling

vllm serve Lna-Lab/Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4 \
    --max-model-len 32768 \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_xml

Docker

docker run --gpus '"device=0"' -p 8016:8016 \
    -v /path/to/model:/models/current:ro \
    --shm-size 16gb \
    vllm/vllm-openai:cu130-nightly \
    vllm serve /models/current --port 8016 --max-model-len 32768 \
    --reasoning-parser qwen3

Benchmark

Single NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM).

Sustained: ~59-61 tok/s (single GPU).

Quantization Details

Recipe

recipe = QuantizationModifier(
    targets=["Linear"],
    ignore=["lm_head", "re:.*visual.*", "re:.*in_proj_a$", "re:.*in_proj_b$"],
    scheme="NVFP4",
)

Calibration

• Dataset: neuralmagic/calibration (LLM split)
• Samples: 512
• Max sequence length: 4096

Reproduction

from transformers import Qwen3_5ForConditionalGeneration, AutoProcessor, AutoTokenizer
from datasets import load_dataset
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier
import torch

MODEL_ID = "huihui-ai/Huihui-Qwopus3.5-27B-v3-abliterated"
OUTPUT = "Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4"

model = Qwen3_5ForConditionalGeneration.from_pretrained(MODEL_ID, dtype="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)

recipe = QuantizationModifier(
    targets=["Linear"],
    ignore=["lm_head", "re:.*visual.*", "re:.*in_proj_a$", "re:.*in_proj_b$"],
    scheme="NVFP4",
)

ds = load_dataset("neuralmagic/calibration", name="LLM", split="train[:512]")

def preprocess(example):
    messages = [
        {"role": m["role"], "content": [{"type": "text", "text": m["content"]}]}
        for m in example["messages"]
    ]
    return processor.apply_chat_template(
        messages, return_tensors="pt", padding=False, truncation=True,
        max_length=4096, tokenize=True, add_special_tokens=False,
        return_dict=True, add_generation_prompt=False,
    )

ds = ds.map(preprocess, batched=False, remove_columns=ds.column_names)

def data_collator(batch):
    assert len(batch) == 1
    return {
        key: (torch.tensor(value) if key != "pixel_values"
              else torch.tensor(value, dtype=torch.bfloat16).squeeze(0))
        for key, value in batch[0].items()
    }

oneshot(
    model=model, recipe=recipe, dataset=ds,
    max_seq_length=4096, num_calibration_samples=512,
    data_collator=data_collator,
)

model.save_pretrained(OUTPUT, save_compressed=True)
processor.save_pretrained(OUTPUT)

Note: No save_mtp_tensors_to_checkpoint needed — Qwopus does not have MTP head (removed during fine-tuning).

Environment

Requirements

• GPU: NVIDIA Blackwell (SM 120)
• VRAM: ~20 GB minimum
• Software: vLLM nightly (cu130)

Notes

• Abliterated (uncensored). Use responsibly.
• Vision tower preserved in BF16.
• No MTP head — speculative decoding not available for this variant.
• NVFP4 is Blackwell-specific. Will not work on Ampere/Hopper.

Credits

• Abliteration: huihui-ai
• Opus distillation: Jackrong/Qwopus3.5-27B-v3
• Original: Qwen
• Quantization recipe: lyf/HauhauCS (proven path)
• Quantization tool: vllm-project/llm-compressor

Support the Base Model Author

• Ko-fi: https://ko-fi.com/huihuiai
• Bitcoin: bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge