license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model: tencent/Hy3
base_model_relation: finetune
tags:

• hy3
• hunyuan
• moe
• text-generation
• abliterated
• obliteratus
• supertune
• nvfp4
• w4a16
• compressed-tensors
• quantized
• vllm

SuperHY3 combines OBLITERATUS 0.1.2 abliteration with a compact,
quality-selected SuperTune post-training update. Both stages are fused into
the checkpoint: there is no adapter to load and no runtime patch to apply.

Release Highlights

What this release is designed to deliver

• Fully fused behavior: OBLITERATUS and SuperTune deltas are already inside
  the weights and match the bundled chat template.
• Targeted editing: only attention output projections were changed; routed
  MoE expert tensors and their NVFP4 packing remain untouched.
• Measured tradeoffs: every benchmark task is published, including the two
  regressions, instead of reporting only the improved scores.
• Release evidence included: benchmark rows, refusal comparison, raw-response
  audit, fusion provenance, and machine-readable release-gate results ship in
  this repository.

Benchmark Snapshot

The comparison uses the same preserved 500 prompts, with 100 prompts per
task, greedy direct decoding, and the same scorer on both sides. IFEval uses the
official Google instruction checker.

Candidate invalid-response, blank-response, and thought-leak ratios were all
0.0 across the 500 items.

The behavioral comparison used an IQ2_M Hy3 runtime with the exact same 40
fused projection deltas and release chat template. It isolates the
post-training behavior, but it is not a native NVFP4-kernel throughput or
perplexity measurement.

OBLITERATUS + SuperTune

1. Direction discovery: OBLITERATUS 0.1.2 built difference-of-means refusal
   directions from 32 paired prompts.
2. Validated abliteration: the final release applies the selected
   attention-output directions to layers 39-77.
3. Quality recovery: a rank-1 update, orthogonalized against the refusal
   direction, is fused into layer 78.
4. Adversarial selection: stronger multi-layer candidates were rejected
   when they introduced stray-script contamination or benchmark loss.
5. Final fusion: 40 self_attn.o_proj tensors were updated; no routed
   expert tensor was changed.

Refusal and Output Integrity

The complete 32-pair OBLITERATUS refusal suite was run against the original and
final runtimes.

Across all 64 candidate responses, automated checks found:

• 0 blank outputs
• 0 special-token leaks
• 0 Unicode replacement characters
• 0 unexpected CJK fragments
• 0 n-gram or character loops
• 0 request errors

A separate 12-case runtime audit passed identity, JSON-only output, tool calls,
no-tool behavior, repetition limits, Korean, defensive security, hidden-prompt
boundaries, gibberish handling, code repair, Hindi, and Kannada.

NVFP4 Build

This release was fused from
kodelow/Hy3-NVFP4-W4A16.
It preserves the source checkpoint's weight-only compressed-tensors layout:
routed experts use NVFP4, while quality-sensitive non-expert paths remain
BF16/F32.

About the Hub parameter badge: packed FP4 weights are represented as U8
storage elements, so the sidebar reports fewer elements than the logical
architecture. The model remains Hy3's 295B-parameter MoE with 21B active
parameters.

The checkpoint is W4A16, so vLLM serves it through the MARLIN NVFP4 path rather
than W4A4 FlashInfer backends.

Serving with vLLM

The approximately 181 GB checkpoint fits a single large-memory accelerator such
as a 275 GB B300, or an appropriately configured multi-GPU deployment.

Single large-memory GPU

vllm serve Jiunsong/SuperHY3-abliterated-NVFP4 \
  --served-model-name superhy3 \
  --tensor-parallel-size 1 \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.90 \
  --load-format auto \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Verified two-node DGX Spark profile

This checkpoint has been served successfully across two 128 GB DGX Spark
systems connected by direct 200 GbE/RoCE, using Ray and tensor parallelism 2.
The verified load placed approximately 84.49 GiB and 84.50 GiB of model
weights on the two ranks.

Both Ray nodes must use the same container or Python environment, an identical
checkpoint, and the same in-container model path. Set the communication
environment on both nodes before starting the Ray workers:

export NCCL_DEBUG=INFO
export NCCL_DEBUG_SUBSYS=INIT,NET,ENV
export NCCL_SOCKET_IFNAME=<200G_INTERFACE>
export GLOO_SOCKET_IFNAME=<200G_INTERFACE>
export NCCL_IB_DISABLE=0

Confirm that ray status reports two nodes and two GPUs, and that the NCCL log
selects NET/IB rather than NET/Socket. Then launch the server from the head
node:

vllm serve /models \
  --served-model-name superhy3 \
  --host 0.0.0.0 \
  --port 8600 \
  --tensor-parallel-size 2 \
  --distributed-executor-backend ray \
  --max-model-len 4096 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype bfloat16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --generation-config vllm \
  --trust-remote-code \
  --enforce-eager \
  --load-format auto

DGX Spark loader note: do not substitute fastsafetensors for auto in
this two-node profile. On the verified Spark setup, fastsafetensors caused
excessive transient unified-memory pressure during loading. The default
loader uses lazy memory-mapped safetensors on local storage and completed the
load reliably. Also avoid the eager safetensors strategy, which reads an
entire file into CPU memory before loading it.

DGX Spark CPU and GPU allocations share one unified-memory pool. Before launch,
stop unrelated inference jobs and stale containers, then verify both nodes:

free -h
swapon --show
nvidia-smi
ray status

If a node becomes unresponsive during loading, inspect the previous boot's
kernel log before changing NCCL settings:

sudo journalctl -b -1 -k --no-pager | \
  grep -Ei 'oom|out of memory|killed process|NVRM|Xid|mlx5|rdma'

• OOM, Out of memory, or Killed process indicates unified-memory pressure.
• NVRM: Xid indicates a GPU, driver, or application fault; retain the Xid
  number and collect sudo nvidia-bug-report.sh.
• An NCCL watchdog timeout that appears only after the peer disappears is
  normally a secondary failure, not the original cause.

For an initial diagnostic boot, MTP can be removed by omitting
--speculative-config. Add it back only after the base server reaches its ready
state. Increase context length and concurrency gradually after startup is
stable. See the
vLLM multi-node deployment guide
and NVIDIA Xid documentation
for deeper diagnostics.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="superhy3",
    messages=[
        {"role": "user", "content": "Explain mixture-of-experts routing."},
    ],
    temperature=0.9,
    top_p=1.0,
    extra_body={
        "chat_template_kwargs": {"reasoning_effort": "no_think"},
    },
)
print(response.choices[0].message.content)

Use reasoning_effort="high" for deeper reasoning and "no_think" for direct
responses. Keep BF16 KV cache on GB10-class hardware; uncalibrated lower-precision
KV cache can amplify stray-token behavior in Hy3 runtimes.

Quantized Editions

Release Integrity

• 99 safetensors shards opened successfully.
• 139,298 indexed tensors matched 139,298 observed tensors.
• 40 tensors were modified across 8 shards.
• 0 routed expert tensors were modified.
• Missing, extra, and wrong-shard tensor counts are all 0.
• The release chat template matches the embedded tokenizer template.
• The automated release gate passed with 0 blockers.
• All 119 Hub files were checksum-verified after upload.

The repository includes the fusion report, release gate, official 500-item
benchmark record, refusal comparison, raw-response audit, OBLITERATUS execution
report, and SuperTune composition reports.

Limitations

• GPQA Diamond and MMLU-Pro are lower than the original in this replay; the
  complete table is retained above.
• Native fused NVFP4 benchmark and long-context throughput measurements were not
  run as part of this release validation.
• Abliteration reduces refusal behavior and can produce content the original
  model would decline. Deployment policy and access control remain the
  operator's responsibility.
• The MLX edition is a separately fused quantized checkpoint, not a conversion
  of these NVFP4 files.

License

Apache-2.0, following the base model and quantized source licenses.