language:
- en
- zh
- es
- fr
- ja
- ko
- ru
license: other
license_name: tencent-hunyuan
license_link: https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE
base_model: tencent/Hy4-flash
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
thumbnail: dealign_mascot.png
extra_gated_prompt: |
By requesting access you confirm you are 18+, will use the model solely in
compliance with applicable law in your jurisdiction, and accept full responsibility
for all outputs. Access is granted at the sole discretion of the repository owner.
extra_gated_fields:
Name: text
Affiliation / organization: text
Country: country
Primary use case: text
I confirm I am 18 or older: checkbox
I accept full responsibility for outputs: checkbox
extra_gated_heading: "Request access — manual approval required"
extra_gated_description: "This model is reviewed manually. Fill in the fields below."
extra_gated_button_content: "Request access"
tags: - tencent
- hunyuan
- hy4
- hy4-flash
- nvfp4
- modelopt
- w4a16
- uncensored
- reasoning
- tool-calling
- moe
[!NOTE]
Please credit if you build on this. If you redistribute, quantize, fine-tune, serve, benchmark, or write about this model, a line like
"Based onlyber-ai/Hy4-flash-UNCENSORED-NVFP4"
in your model card or write-up means a lot. Thank you.
[!IMPORTANT]
NVFP4 (modelopt) weight-only · group_size=16 · F8_E4M3 per-group scales ·
F32 globalweight_scale_2· dragonfly / vision / engram / HC paths kept BF16/F32.
219 GB disk (vs 411 GB BF16 / 267 GB FP8). Produced via direct-shards quantization
in Nvidia modelopt NVFP4 format.
[!WARNING]
Serving note (as of 2026-11-21). Publicvllm/vllm-openai:hy4-flash(2026-11-20),lmsysorg/sglang:hy4(2026-11-20), andlmsysorg/sglang:nightly-cu134-20261121
images do not yet fully support serving NVFP4 Hy4-flash. In sglang,--quantization modelopt_fp4wires up the MoE-experts path (routedw13/w2), but the
non-MoE Linears used by Hy4-flash — attentionq_b_proj/kv_b_proj/ fusedqkv_a_proj_with_mqa, the dense MLP at layer 0, and (before patching) theshared_expertsblock — are instantiated without NVFP4 parameter registration,
so theirweight_scale/weight_scale_2destinations don't exist at load. The
artifact is structurally identical to
atakans/Hy4-flash-NVFP4
(same tensor naming, shapes, and dtypes), so it will serve the moment upstream
SGLang / vLLM land NVFP4 parameter registration for Hy4-flash's full Linear set.
Today, use the FP8 variantlyber-ai/Hy4-flash-UNCENSORED-FP8
for direct SGLang / vLLM serving, or run these NVFP4 shards through TensorRT-LLM
(Nvidia's native modelopt NVFP4 serve stack).
Hy4-flash — UNCENSORED (NVFP4)
Uncensored release · NVFP4 modelopt · 128K context · Extended thinking · Tools
What this is
This is tencent/Hy4-flash (Hunyuan V4
Flash) with refusal behavior removed, then quantized to NVFP4 in Nvidia modelopt
format. All upstream capabilities — extended thinking with reasoning_effort tiers,
tool calling, DSA attention, dragonfly MTP fusion, 128K+ context — are preserved.
Quantization details
- Format: modelopt NVFP4,
hf_quant_config.jsonat root (quant_algo: NVFP4,group_size: 16), compatible with upstream Nvidia modelopt / TensorRT-LLM and
(pending full support) vLLM--quantization modelopt_fp4+ SGLang--quantization modelopt_fp4. - FP4 e2m1 levels:
{±0, ±0.5, ±1, ±1.5, ±2, ±3, ±4, ±6}(16 values). - Scaling (per weight tensor):
.weight: uint8, 2 FP4 values per byte, shape(out, in/2).weight_scale: F8_E4M3, per-group (16-wide) scale, shape(out, in/16).weight_scale_2: F32 scalar (global amax renormalization so per-group scales stay inside F8_E4M3 range).input_scale: F32 scalar = 1.0 (weight-only NVFP4; placeholder for activation path)
- Dequantization:
W_fp32 = packed_fp4 × weight_scale.to(fp32) × weight_scale_2 - MoE experts: per-expert, unfused (
experts.{i}.gate_proj,experts.{i}.up_proj,experts.{i}.down_proj) matching upstream NVFP4 conventions; 512 experts × 34 MoE
layers. - Excluded from quantization (kept BF16/F32):
lm_head, allmodel.dfly_layers.*
(dragonfly MTP fusion),vit.*(vision tower),engram_layer.*(Engram speculative
memory),self_attn.compressed_indexer.*,hc_pre / hc_attn / hc_mlphyperconnection
parameters,learnable_sink_param, routergate.weight. - Shard count: 148. Total size: 219 GB.
Reference ancestry
- Refusal behavior removed from the BF16 counterpart of
tencent/Hy4-flash(same
method and strengths used in the FP8 companion releaselyber-ai/Hy4-flash-UNCENSORED-FP8). - NVFP4 format matches
atakans/Hy4-flash-NVFP4
(producer: modelopt 0.48.0, group_size 16, same tensor naming and dtypes).
FP8-counterpart benchmarks (same base crack, served as FP8)
These are the measured gates on the shipped FP8 variant, which is produced from the
same BF16-cracked base this NVFP4 quantizes. NVFP4-specific re-measurement will be
added here once serving support lands.
| gate | result |
|---|---|
HB-320 full high COMPLY |
306/320 (95.6%) (content 256 + reasoning-engaged 50) |
HB-320 full high borderline |
14/320 (4.4%) |
HB-320 full no_think COMPLY |
313/320 (97.8%) |
MMLU-228 no_think |
87.28% (base 88.16% → Δ −0.88 pp) |
MMLU-228 high |
91.67% (base 94.30% → Δ −2.63 pp) |
| harsh-MT preserved (3 seeds × 10 turns × 49K) | 0 loops / 0 no_answer / mean_entropy ~0.30 nats |
| harsh-MT content (3 seeds × 10 turns × 49K) | 0 loops / 0 no_answer / mean_entropy ~0.30 nats |
| 30-turn stress preserved (2 seeds × 30 turns) | 0 errors, 40K+ context |
| loop_check benign (72 × 2 modes) | 4 loops no_think + 2 loops high (2.8% residual) |
How to serve (when supported)
TensorRT-LLM (Nvidia modelopt native)
# Build engine from modelopt NVFP4 checkpoint
trtllm-build \
--checkpoint_dir /path/to/Hy4-flash-UNCENSORED-NVFP4 \
--output_dir /path/to/engine \
--gemm_plugin fp4 --gpt_attention_plugin bfloat16 \
--max_batch_size 32 --max_input_len 8192 --max_output_len 24576
vLLM (pending NVFP4 Hy4-flash support)
docker run -it --rm --gpus all --ipc=host --shm-size=64g \
-v /path/to/model:/model \
vllm/vllm-openai:latest \
--model /model \
--served-model-name hy4-flash \
--tensor-parallel-size 8 \
--max-model-len 131072 \
--quantization modelopt_fp4 \
--trust-remote-code \
--host 0.0.0.0 --port 8000
SGLang (pending NVFP4 Hy4-flash support)
docker run -d --gpus all --ipc=host --shm-size=64g \
-v /path/to/model:/model \
-p 8000:8000 \
--entrypoint python3 \
lmsysorg/sglang:hy4 \
-m sglang.launch_server \
--model-path /model --served-model-name hy4-flash \
--tp-size 8 --quantization modelopt_fp4 --port 8000
Hardware requirements (NVFP4)
| GPU | Setup | Status |
|---|---|---|
| 4× B300 SXM6 | TP4 | ✅ Fits (219 GB in 564 GB VRAM) |
| 8× H200 141GB | TP8, FP4 kernels | ✅ Fits |
| 8× H100 80GB | TP8, FP4 kernels | ✅ Fits |
| 2× B300 | TP2 | ⚠️ Tight, context-window dependent |
Reasoning + tools
chat_template_kwargs = {"reasoning_effort": "high"} # or "no_think"
Responsible use
You are solely responsible for how you use this model and its outputs.
- Do not use for generating illegal content in your jurisdiction
- Do not use to harm, deceive, or exploit real individuals
- Do not use to generate CSAM or non-consensual intimate content
- Comply with all applicable laws and regulations
License
Inherits the Tencent Hy License, Version 1.0
from the base model. The modification does not change the license terms.
Credits
- Base model: tencent/Hy4-flash
- Released by: Lyber AI × dealign.ai
