← back to catalog · registered 2026-11-18 09:04

disillusioned/Huihui-GLM-4.7-Flash-abliterated-NVFP4

disillusioned Glm MoE second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/disillusioned%2FHuihui-GLM-4.7-Flash-abliterated-NVFP4"
Response includes
  • classification m1
  • files 16
  • benchmarks 11 entries
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M1
Primary method

Direct removal

No other method signals detected in this model.
Confidence
MEDIUM
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=0 (base model)
  • no specific method indicators - defaulting to M1 (most common)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
MEDIUM
Why we say so
primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)
Downloads · 30-day
285
Likes
0
Model age
4d ago
created 2026-11-18
Downloads over time
Now 285 → from 0 ↑ 0%
0105209314 0 on Nov 18 285 on Nov 22 285 on Nov 21 Nov
Nov 18 → Nov 22 · 5 snapshots · spans 4 days

Benchmarks

Benchmark Score Source
Entertainment 1.4 UGI
Hazardous 0.6 UGI
Natural Intelligence 18.32 UGI
Political lean -10.3% UGI
Sensitive-Info 12.1 UGI
SocPol 1.5 UGI
UGI 31.4 UGI
Willingness (10) 7 UGI
W10-Adherence 7 UGI
W10-Direct 7 UGI
Writing 25.08 UGI

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
mit
Languages
en zh
Tags
transformers safetensors glm4_moe_lite text-generation abliterated uncensored nvfp4 fp4 4-bit compressed-tensors llm-compressor vllm

Related

Total size
20.4 GB
Files
16
Quantizations
1
Registered
2026-11-18 09:04
Last updated on HF
2026-11-18 08:18

Files by quantization

Auxiliary files 16 files 20.4 GB
model-00002-of-00004.safetensors 4.66 GB 368f035d download
model-00001-of-00004.safetensors 4.66 GB f60207b2 download
model-00003-of-00004.safetensors 4.66 GB 67c8f1d5 download
model-00004-of-00004.safetensors 4.03 GB d5e6ca5a download
model_mtp.safetensors 2.38 GB e75debcd download
tokenizer.json 19.3 MB 19e77364 download
model.safetensors.index.json 3.47 MB 3ecdbb37 download
config.json 24.3 KB ba9afc59 download
README.md 11.2 KB 98ddddc7 download
chat_template.jinja 3.05 KB 2ab98ef0 download
.gitattributes 1.53 KB 52373fe2 download
LICENSE 1.05 KB d8e3f343 download
NOTICE 832 B bc2e2917 download
tokenizer_config.json 749 B 03b937c6 download
recipe.yaml 497 B 149e3932 download
generation_config.json 177 B 8265972b download

README current version from Hugging Face


library_name: transformers
pipeline_tag: text-generation
license: mit
language:

  • en
  • zh
    base_model:
  • huihui-ai/Huihui-GLM-4.7-Flash-abliterated
    base_model_relation: quantized
    tags:
  • abliterated
  • uncensored
  • nvfp4
  • fp4
  • 4-bit
  • compressed-tensors
  • llm-compressor
  • vllm
  • quantized

disillusioned/Huihui-GLM-4.7-Flash-abliterated-NVFP4

NVFP4 (4-bit) quantization of huihui-ai/Huihui-GLM-4.7-Flash-abliterated.

The weights of huihui-ai's abliterated GLM-4.7-Flash quantized to the NVFP4 format
(packed 4-bit float, group size 16, FP8-E4M3 scales) with
llm-compressor /
compressed-tensors, for efficient serving on
NVIDIA Blackwell GPUs (sm_120, e.g. RTX 5090) with vLLM.
Only the weights are quantized — the architecture, tokenizer, and chat template are unchanged from
the source model, and the MTP speculative draft head (model_mtp.safetensors) is preserved.

Serving with vLLM

vllm serve disillusioned/Huihui-GLM-4.7-Flash-abliterated-NVFP4 \
  --reasoning-parser glm47 \
  --enable-auto-tool-choice --tool-call-parser glm47 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

The NVFP4 format is read automatically from the model's quantization_config (compressed-tensors);
no extra flag is needed to enable it.


Below is the original model card from huihui-ai.

huihui-ai/Huihui-GLM-4.7-Flash-abliterated

This is an uncensored version of zai-org/GLM-4.7-Flash created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

ollama

A new version is being uploaded. Please download it again.
Please use the latest version of ollama 0.15.1

You can use huihui_ai/glm-4.7-flash-abliterated directly,

ollama run huihui_ai/glm-4.7-flash-abliterated

Usage

You can use this model in your applications by loading it with Hugging Face's transformers library:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import argparse
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TextStreamer
import torch
import os
import signal
import time

def parse_args():
    parser = argparse.ArgumentParser(
        description="Merge LoRA weights into huihui-ai/Huihui-GLM-4.7-Flash-abliterated base model and save the full model."
    )
    parser.add_argument(
        "--base_model",
        type=str,
        default="huihui-ai/Huihui-GLM-4.7-Flash-abliterated",
        help="HuggingFace repo or local path of the base model.",
    )
    parser.add_argument(
        "--dtype",
        type=str,
        default="bfloat16",
        choices=["float16", "bfloat16", "float32"],
        help="Data type for loading the base model (default: bfloat16).",
    )
    parser.add_argument(
        "--device_map",
        type=str,
        default="auto",
        help="Device map for model loading (e.g. 'cpu', 'auto').",
    )
    return parser.parse_args()

def main():
    cpu_count = os.cpu_count()
    print(f"Number of CPU cores in the system: {cpu_count}")
    half_cpu_count = cpu_count // 2
    os.environ["MKL_NUM_THREADS"] = str(half_cpu_count)
    os.environ["OMP_NUM_THREADS"] = str(half_cpu_count)
    torch.set_num_threads(half_cpu_count)

    print(f"PyTorch threads: {torch.get_num_threads()}")
    print(f"MKL threads: {os.getenv('MKL_NUM_THREADS')}")
    print(f"OMP threads: {os.getenv('OMP_NUM_THREADS')}")

    args = parse_args()

    # Load the model and tokenizer
    print(f"Load Model {args.base_model} ... ")
    quant_config_4 = BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.bfloat16,
        bnb_4bit_quant_type="nf4" if args.device_map == "cpu" else "fp4",
        bnb_4bit_use_double_quant=True,
        llm_int8_enable_fp32_cpu_offload=True,
    )

    torch_dtype = {
        "float16": torch.float16,
        "bfloat16": torch.bfloat16,
        "float32": torch.float32,
    }[args.dtype]

    model = AutoModelForCausalLM.from_pretrained(
        args.base_model,
        dtype=torch_dtype,
        device_map=args.device_map,
        trust_remote_code=True,
        #low_cpu_mem_usage=True,
    )

    tokenizer = AutoTokenizer.from_pretrained(args.base_model, trust_remote_code=True)

    messages = []
    skip_prompt=True
    skip_special_tokens=True

    class CustomTextStreamer(TextStreamer):
        def __init__(self, tokenizer, skip_prompt=True, skip_special_tokens=True):
            super().__init__(tokenizer, skip_prompt=skip_prompt, skip_special_tokens=skip_special_tokens)
            self.generated_text = ""
            self.stop_flag = False
            self.init_time = time.time()  # Record initialization time
            self.end_time = None  # To store end time
            self.first_token_time = None  # To store first token generation time
            self.token_count = 0  # To track total tokens

        def on_finalized_text(self, text: str, stream_end: bool = False):
            if self.first_token_time is None and text.strip():  # Set first token time on first non-empty text
                self.first_token_time = time.time()
            if stream_end:
                self.end_time = time.time()  # Record end time when streaming ends

            self.generated_text += text
            self.token_count += 1
            print(text, end="", flush=True)

            if self.stop_flag:
                raise StopIteration

        def stop_generation(self):
            self.stop_flag = True
            self.end_time = time.time()  # Record end time when generation is stopped

        def get_metrics(self):
            """Returns initialization time, first token time, first token latency, end time, total time, total tokens, and tokens per second."""
            if self.end_time is None:
                self.end_time = time.time()  # Set end time if not already set
            total_time = self.end_time - self.init_time  # Total time from init to end
            tokens_per_second = self.token_count / total_time if total_time > 0 else 0
            first_token_latency = (self.first_token_time - self.init_time) if self.first_token_time is not None else None
            metrics = {
                "init_time": self.init_time,
                "first_token_time": self.first_token_time,
                "first_token_latency": first_token_latency,
                "end_time": self.end_time,
                "total_time": total_time,  # Total time in seconds
                "total_tokens": self.token_count,
                "tokens_per_second": tokens_per_second
            }
            return metrics

    def generate_stream(model, tokenizer, messages, skip_prompt, skip_special_tokens, max_new_tokens):
        inputs = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            return_dict=True,
            return_tensors="pt",
        ).to(model.device)

        streamer = CustomTextStreamer(tokenizer, skip_prompt=skip_prompt, skip_special_tokens=skip_special_tokens)

        def signal_handler(sig, frame):
            streamer.stop_generation()
            print("\n[Generation stopped by user with Ctrl+C]")

        signal.signal(signal.SIGINT, signal_handler)

        print("Response: ", end="", flush=True)
        try:
            generated_ids = model.generate(
                **inputs,
                max_new_tokens=max_new_tokens,
                streamer=streamer
            )
            del generated_ids
        except StopIteration:
            print("\n[Stopped by user]")

        del inputs
        torch.cuda.empty_cache()
        signal.signal(signal.SIGINT, signal.SIG_DFL)

        return streamer.generated_text, streamer.stop_flag, streamer.get_metrics()

    while True:
        user_input = input("User: ").strip()
        if user_input.lower() == "/exit":
            print("Exiting chat.")
            break
        if user_input.lower() == "/clear":
            messages = []
            print("Chat history cleared. Starting a new conversation.")
            continue
        if user_input.lower() == "/skip_prompt":
            if skip_prompt:
                skip_prompt = False
                print("skip_prompt = False.")
            else:
                skip_prompt = True
                print("skip_prompt = True.")
            continue
        if user_input.lower() == "/skip_special_tokens":
            if skip_special_tokens:
                skip_special_tokens = False
                print("skip_special_tokens = False.")
            else:
                skip_special_tokens = True
                print("skip_special_tokens = True.")
            continue
        if not user_input:
            print("Input cannot be empty. Please enter something.")
            continue

        messages.append({"role": "user", "content": user_input})
        response, stop_flag, metrics = generate_stream(model, tokenizer, messages, skip_prompt, skip_special_tokens, 40960)
        print("\n\nMetrics:")
        for key, value in metrics.items():
            print(f"  {key}: {value}")

        print("", flush=True)

        if stop_flag:
            continue
        messages.append({"role": "assistant", "content": response})

if __name__ == "__main__":
    main()

Usage Warnings

  • Risk of Sensitive or Controversial Outputs: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.

  • Not Suitable for All Audiences: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.

  • Legal and Ethical Responsibilities: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.

  • Research and Experimental Use: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.

  • Monitoring and Review Recommendations: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.

  • No Default Safety Guarantees: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.

Donation

Your donation helps us continue our further development and improvement, a cup of coffee can do it.
  • bitcoin:
  bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge
  • Support our work on Ko-fi!

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-18 Add NVFP4 model card (based on huihui-ai card) b2d0651 11.2 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration