license: mit
tags: [abliterated, int8, autoround, safetensors]

Qwen3.5-0.8B-Abliterated-INT8

True INT8 quantized version of abliterated Qwen3.5-0.8B using AutoRound (no wrappers needed).

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_DIR = "Heouzen/Qwen3.5-0.8B-Abliterated-INT8"

print("Memuat tokenizer dan model True INT8...")
tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR, trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_DIR,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

prompt = "Halo, jelaskan secara singkat siapa kamu."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

print(f"\nPrompt: {prompt}\n")
print("Generating response...")
outputs = model.generate(
    **inputs, 
    max_new_tokens=150, 
    do_sample=True, 
    temperature=0.7
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)