YifeiDevs/Huihui-Qwen3-8B-abliterated-v2-FP8-Comfy

library_name: transformers
tags:

• fp8
• qwen

ComfyUI

Load CLIP

• clip_name: Huihui-Qwen3-8B-abliterated-v2-FP8.safetensors
• type: flux2
• device: default

Disclaimer:

FP8 quant of Huihui-Qwen3-8B-abliterated-v2. Safety refusals have been removed (abliterated). User assumes all legal responsibility for its use and output.

FP8 Conversion Script

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "huihui-ai/Huihui-Qwen3-8B-abliterated-v2"
SAVE_DIR = "Huihui-Qwen3-8B-abliterated-v2-FP8-Comfy"

print("Loading model...")
# Load as BF16, convert on CPU to save VRAM
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID, 
    torch_dtype=torch.bfloat16, 
    device_map="cpu",
    trust_remote_code=True
)

print("Converting to FP8 (E4M3)...")
# Iterate through parameters and convert to float8_e4m3fn
for name, param in model.named_parameters():
    # Only convert Linear layer weights, skip LayerNorm and Bias for precision
    if "weight" in name and param.ndim == 2: 
        param.data = param.data.to(torch.float8_e4m3fn)

print(f"Saving to {SAVE_DIR}...")
# Save with safetensors, max 20GB per file
model.save_pretrained(
    SAVE_DIR, 
    safe_serialization=True,
    max_shard_size="10GB"
)
print("Done!")