library_name: transformers
license: apache-2.0
language:

• en
• zh
  tags:
• qwen
• qwen3.8
• nvfp4
• qat
• modelopt
• blackwell
• vllm
• agentic
• coding
• reasoning
• abliterated
• uncensored
• huihui
• 4-bit
  base_model:
• huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated
  base_model_relation: quantized
  datasets:
• nvidia/OpenCodeInstruct
  pipeline_tag: image-text-to-text

⚡ Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated (NVFP4 Scale-QAT)

High-Performance Agentic Coding & Reasoning Model by KatterMobile (Kaden)

❤️ Please leave a LIKE / HEART on this Hugging Face repository!
If you find this model, the Scale-QAT quantization-aware training, or the NVFP4 export useful, hitting the heart button at the top of this page makes a huge difference and helps independent developers like me stay motivated to release more frontier open-source models for everyone.

☕ A Personal Note from KatterMobile

"Hello! I'm KatterMobile (real name Kaden). I spent hours calibrating, running Quantization-Aware Training (Scale-QAT), verifying, and exporting this model on a single NVIDIA GB10 Grace Blackwell DGX Spark workstation. I am an independent researcher and developer without corporate backing. If you find this release valuable or use it in your coding workflows, please consider supporting me on Ko-fi—literally anything helps cover compute electricity and hosting costs! Thank you so much for reading and supporting independent AI."

☕ Support on Ko-fi: https://ko-fi.com/kattermobile
📬 Support & Inquiries: [email protected]

🌟 Model Overview

Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-NVFP4-QAT is a native NVFP4 (W4A4) + FP8 KV Cache Quantization-Aware Trained (Scale-QAT) build of huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated (commit 14c13fcf593d692a5b96f29afecca6fe2fc030e6), itself an abliterated (uncensored) version of nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2.

Rather than relying on one-shot Post-Training Quantization (PTQ) alone, this checkpoint was trained with NVIDIA ModelOpt 0.48 (09a14b9ae89522363ffca9b0ac2245cbbb073fdd) using the learned-scale QAT recipe general/qad/nvfp4_lsq-mse_init-fp8_kv across 22,000 execution-filtered nvidia/OpenCodeInstruct tasks (average_test_score=1 with all-pass unit test execution records).

• Significant Loss Reduction over Initial Quantization: Held-out 64-example test loss dropped from 0.265642 (initial MSE + FP8 scale sweep) down to 0.158151 after Scale-QAT (-40.5% test loss, validation loss 0.241453 -> 0.136581 at best step 100).
• Full Composite Architecture Preserved: All 1,199 source weight keys are preserved, including 348 BF16 Vision Tower (27 ViT blocks) & Multi-Token Prediction (MTP) tensors alongside 400 packed NVFP4 language projection weights, packaged into 3 safetensors shards (~20.59 GB).
• Hardware-Verified on NVIDIA GB10 Grace Blackwell: Passed full-model offline vLLM 0.33.0rc1 (CUDA 13.4) smoke verification using the native modelopt_fp4 backend.

⚙️ Mandatory Thinking & Sampling Parameters

[!IMPORTANT]
Preserve Reasoning Traces: To obtain full reasoning performance and prevent premature cutoff, ensure your client or UI preserves <think> ... </think> tokens and exposes them in reasoning_content.

🧠 Transparent Architecture: Exactly How I Trained & Quantized It

[huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated (BF16, rev 14c13fcf)]
                                       │
                                       ▼
          [MSE + FP8 Scale-Sweep Initialization (512 Calibration Samples)]
                                       │
                                       ▼
      [NVIDIA ModelOpt NVFP4 Learned-Scale QAT (general/qad/nvfp4_lsq-mse_init-fp8_kv)]
   (1,520,435,200 trainable block-scale parameters, frozen BF16 weights, 22k OpenCodeInstruct)
                                       │
                                       ▼
         [Exact Tied-LSQ Scale Restoration & 3-Shard NVFP4 Safetensors Export]
  (400 packed W4A4 language projections + 348 BF16 Vision/MTP tensors, 1,199 keys preserved)
                                       │
                                       ▼
          [vLLM 0.33.0rc1 Verification on NVIDIA GB10 (CUDA 13.4, modelopt_fp4)]
      (Passed arithmetic, structured JSON, and Python code generation smoke suite)

1. The Huihui Coder390 EfficientThink Foundation

• Source Checkpoint: huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated (revision 14c13fcf593d692a5b96f29afecca6fe2fc030e6).
• Lineage: Uncensored (layers 2–52 abliterated) derivative of nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2, combining efficient chain-of-thought reasoning with strong software engineering synthesis.

2. Execution-Verified Coding Calibration & QAT Dataset

• Dataset: nvidia/OpenCodeInstruct (revision 8f3ba5bafe4d6e8db46082cf7ae6741bc370604d, CC-BY-4.0).
• Strict Filtering: Examined 62,121 samples and selected 22,000 unique prompts (deduplicated by SHA-256) requiring average_test_score=1, unit_tests present, and nonempty all-pass recorded execution statuses (seed=42).
• Split: 19,800 training / 1,100 validation / 1,100 test rows, with fixed 64-example evaluation and test subsets and a 4,096-token sequence cap with dynamic right-padding trimming (rounded to 128-token multiples). All tokenized rows retain assistant supervision.

3. NVFP4 W4A4 Learned-Scale QAT (general/qad/nvfp4_lsq-mse_init-fp8_kv)

• Why Scale-QAT? Full-weight Adam QAT on a 27B hybrid model exceeds single-workstation memory. Scale-QAT freezes the BF16 base weights and directly optimizes all 1,520,435,200 tied NVFP4 block-scale parameters (group_size=16) initialized via MSE and an FP8 scale sweep over 512 calibration samples.
• Training Configuration: Official QATTrainer, learning rate 1e-4, micro-batch size 1 with gradient accumulation 4, up to 200 optimizer steps (20,750.6s total wall time, 74.59 GiB peak CUDA allocated on GB10), evaluating every 50 steps with plateau stopping.
• Measured Loss Progression (metrics.json):

(Note: These are teacher-forced cross-entropy loss measurements on the held-out OpenCodeInstruct splits, not broad external coding benchmark scores.)

4. Precision Exclusions & Exact Scale Export

• Quantized Modules (NVFP4 W4A4, group_size=16): 400 eligible language model linear projections.
• KV Cache: Quantized to FP8 (kv_cache_quant_algo: "FP8").
• Higher-Precision Exclusions (Retained in BF16): Sensitive linear-attention gates and convolutions (conv1d, in_proj_a, in_proj_b), token embeddings (embed_tokens), lm_head, all 27 Vision Tower ViT blocks (visual.*), and the Multi-Token Prediction (mtp.*) draft head (348 retained tensors verified bit-for-bit).
• Exact Scale Export Verification: Export restored all 1,520,435,200 saved scale parameters from step 100 (best_scales.pt). The 90 raw negative learned scale parameters across 6 quantizers derive the exact same clamped FP8-rounded physical scales during export as during fake-quant training, verified via real CUDA regression tests (scale_diagnostics.json).

🚀 Serving with vLLM (1M Context & 512k Output)

Verified on vLLM 0.33.0rc1.dev806+ga2bcb5657.d20261105 (vllm/vllm-openai CUDA 13.4 nightly on NVIDIA GB10) with --quantization modelopt_fp4:

vllm serve KatterMobile/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-NVFP4-QAT \
  --served-model-name Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-NVFP4-QAT \
  --quantization modelopt_fp4 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 1048576 \
  --gpu-memory-utilization 0.88 \
  --max-num-seqs 10 \
  --max-num-batched-tokens 16384 \
  --enable-prefix-caching \
  --max-cudagraph-capture-size 32 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}' \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --default-chat-template-kwargs '{"enable_thinking": true, "reasoning_effort": "xhigh", "preserve_thinking": true}' \
  --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' \
  --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0, "presence_penalty": 0.0, "repetition_penalty": 1.0, "max_new_tokens": 524288}'

(For lightweight local testing without 1M YaRN override, omit --hf-overrides to use the native 262,144 context window or pass --max-model-len 32768.)

💻 Quick Start with Pi Code CLI (pi)

Add to ~/.pi/agent/models.json:

{
  "providers": {
    "vllm-home": {
      "baseUrl": "http://<YOUR_HOST_IP>:8000/v1",
      "api": "openai-completions",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": true,
        "thinkingFormat": "qwen-chat-template",
        "chatTemplateKwargs": {
          "enable_thinking": true,
          "preserve_thinking": true
        }
      },
      "models": [
        {
          "id": "Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-NVFP4-QAT",
          "name": "Huihui Qwen3.8 27B Coder390 EfficientThink NVFP4 QAT",
          "reasoning": true,
          "contextWindow": 1048576,
          "maxTokens": 524288,
          "samplingParams": {
            "temperature": 1.0,
            "top_p": 0.95,
            "top_k": 20
          }
        }
      ]
    }
  }
}

Then run in your terminal:

pi --provider vllm-home --model Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-NVFP4-QAT

📦 Included Reproducibility & Audit Artifacts

To ensure complete transparency, the repository includes the machine-readable verification and provenance records generated during the GX10 Scale-QAT pipeline:

• metrics.json — BF16, initialized-quant, and best Scale-QAT (step 100) eval/test losses, runtime, and CUDA memory telemetry.
• provenance.json — Exact source SHA-256 digests, NVIDIA/Model-Optimizer commit (09a14b9ae89522363ffca9b0ac2245cbbb073fdd), package versions (torch 2.15.0, transformers 5.18.0, nvidia-modelopt 0.48.0rc1.dev323+g09a14b9ae), and recipe parameters.
• data_manifest.json — nvidia/OpenCodeInstruct revision (8f3ba5bafe4d6e8db46082cf7ae6741bc370604d), filter criteria, and sample counts.
• quantizers.json & scale_diagnostics.json — Per-layer quantizer configuration and learned scale distribution diagnostics.
• vllm_smoke.json & installation.json — Offline vLLM 0.33.0rc1 (modelopt_fp4) artifact integrity and smoke test verification output on NVIDIA GB10.

⚠️ Usage Warnings

• Uncensored / Abliterated Base Model: Because the upstream base model (huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated) had refusal directions abliterated across layers 2–52, safety filtering is significantly reduced. Users are solely responsible for reviewing outputs and ensuring compliance with applicable laws and safety requirements.

📬 Contact, Support & Feedback

• ☕ Ko-fi Donations: https://ko-fi.com/kattermobile
• 📧 Support & Bug Reports: [email protected]
• 💬 Community Discussions: Feel free to open an issue or discussion thread right here on Hugging Face!

📜 Credits, Attribution & Licensing

• Huihui-AI: For the abliterated source checkpoint (huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated).
• Nerkyor: For the merged coding & reasoning foundation (nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2).
• Qwen Team (Alibaba Cloud): For the foundational Qwen architecture.
• NVIDIA ModelOpt & OpenCodeInstruct Authors:
  • Quantized using NVIDIA TensorRT Model Optimizer (Model-Optimizer).
  • Calibration and Scale-QAT data sourced from NVIDIA OpenCodeInstruct (CC-BY-4.0): Wasi Uddin Ahmad, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Vahid Noroozi, Somshubra Majumdar, and Boris Ginsburg, "OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs" (2025), arXiv:2504.04030.

Licensed under the Apache 2.0 License.