license: apache-2.0
base_model:
- huihui-ai/Huihui-Qwen3.6-27B-abliterated
pipeline_tag: text-generation
library_name: transformers
tags: - agentic
- tool-calling
- nvfp4
- qwen3.6
- modelopt
PINQWEN-3.6-27B-NVFP4-ABLITERATED
Blackfrost internal archive. Agentic / tool-calling specialist — Qwen3.6-27B fine-tuned on the The Void (v3) corpus, then quantized to NVFP4 (nvidia-modelopt) for fast serving on Blackwell (SM120).
At a glance
| Field | Value |
|---|---|
| Base | huihui-ai/Huihui-Qwen3.6-27B-abliterated (Qwen3.6-27B — hybrid Gated-DeltaNet + attention, abliterated) |
| Params | ~27 B (hybrid dense, 64-layer + 1 MTP) |
| Fine-tune data | The Void (v3) — 5,025 records (3,132 knowledge/CoT + 1,893 agentic ReAct, 38% agentic) |
| Method | unsloth QLoRA, rank 32, α 32, all 7 proj targets, multi-turn masking, length-grouped |
| Schedule | 3 epochs, 474 steps, lr 2e-4 linear, bsz 2 × accum 2 × 8 GPU (eff 32) |
| Hardware | 8× RTX PRO 6000 Blackwell (g4-standard-384), ~1.5 h wall |
| Quant | NVFP4 (nvidia-modelopt NVFP4_DEFAULT_CFG), text-only, MTP grafted |
| Serves on | vLLM, Blackwell SM120 |
| Trained | 2026-07-08 · Blackfrost-AI |
Benchmark — agent_benchmark.py (vs. base, identical battery)
| Metric | Base (abliterated) | VEGA-27B |
|---|---|---|
| Tool-calling accuracy | 7 / 8 | 8 / 8 |
| Avg latency / response | 10.5 s | 6.2 s (−41%) |
| Avg tokens / response (verbosity) | 267 | 158 (−41%) |
| Multi-turn drivability | 2 / 2 | 1 / 2 (mock-tool keyword artifact — not a real regression; needs a live tool to score) |
Takeaway: the The Void (v3) training made an already-strong base more accurate at tool selection, ~40% faster, and ~40% more concise.
Quantization details (why this one serves, unlike a naive convert)
- Tool:
nvidia-modelopt(NVFP4_DEFAULT_CFG) — the fast SM120 path.llm-compressor/compressed-tensors NVFP4 null-outputs on SM120 + vLLM for this hybrid family, so modelopt is required. - Arch kept:
Qwen3_5ForConditionalGeneration+language_model_only: true. vLLM has no text-only class for the Qwen3.5 hybrid family, so the full class + this flag is how a text model loads. - Kept in bf16 (quant ignore list):
linear_attn.conv1d(the Gated-DeltaNet conv),lm_head, vision tower, MTP head. - MTP head (15 tensors) grafted back in bf16 → working speculative-decoding draft path.
- Vision tower stripped → text-only, smaller + faster than the VLM variant.
- Recipe:
lna-lab/GGUF-to-NVFP4-SM120qwen36_27b_text_mtp.py.
Serving (vLLM on Blackwell)
vllm serve Blackfrost-AI/VEGA-27B-NVFP4 \
--trust-remote-code \
--enable-auto-tool-choice --tool-call-parser qwen3_xml \
--max-model-len 131072
No --quantization flag — vLLM auto-detects the modelopt NVFP4 checkpoint. Tool calls use Qwen3's XML format (<function=name><parameter=x>…), parsed by qwen3_xml.