language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags: - qwen
- abliteration
- uncensored
- nf4
- quantized
- 14b
base_model: Qwen/Qwen3-14B
widget: - example_title: Example
example_inputs: "Hello, how are you?"
model-index: - name: Qwen3-14B-Abliterated-v2-nf4
results: []
derived_from: huihui-ai/Huihui-Qwen3-14B-abliterated-v2
Qwen3-14B-Abliterated-v2-nf4
Model Overview
This repository contains a quantized version (NF4, using BitsAndBytes) of the Qwen3-14B-Abliterated-v2 model.
The original model is an uncensored variant of Qwen/Qwen3-14B, created using the abliteration technique to remove refusal behaviors (see remove-refusals-with-transformers).
This quantization was performed by ikarius to reduce model size and enable efficient inference on consumer hardware, while preserving the uncensored capabilities of the base model.
Key Features:
- Base Model: Qwen3-14B (abliterated for uncensoring)
- Version: Abliterated-v2 (improved over v1)
- Quantization: NF4 (4-bit NormalFloat via BitsAndBytes)
- Parameters: 14 Billion
- License: Refer to the original Qwen3 license (Apache 2.0 with additional terms); abliteration does not alter the license.
- Intended Use: Research, experimentation, and creative applications.
Warning: This model is uncensored and may generate sensitive or harmful content—use responsibly.
Installation
- Install the required dependencies:
pip install transformers torch bitsandbytes accelerate
Ensure you have a compatible CUDA setup for GPU acceleration.
(Optional) For CPU-only inference:bash
pip install optimum[exporters]
Usage
Load and run the model using Hugging Face Transformers.
Python Example
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
device_map="auto",
trust_remote_code=True,
)
Example inference
prompt = "Hello, how are you?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=128,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
Inference Tips
VRAM: ~8–10 GB required for 14B NF4 on a single GPU.
Batch Size: Start with 1.
Thinking Mode: v2 supports step-by-step reasoning prompts.
Streaming: Use TextStreamer for real-time output.
Quantization Details
Method: BitsAndBytes NF4 (normal float 4-bit)
Quantizer: ikarius
Benefits: ~75% size reduction vs BF16, minimal quality loss
Trade-offs: Slight perplexity increase
Reproduce Quantization
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch
original_model = AutoModelForCausalLM.from_pretrained(
"huihui-ai/Huihui-Qwen3-14B-abliterated-v2",
torch_dtype=torch.bfloat16,
device_map="auto",
)
quant_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
quantized_model = AutoModelForCausalLM.from_pretrained(
"huihui-ai/Huihui-Qwen3-14B-abliterated-v2",
quantization_config=quant_config,
device_map="auto",
)
quantized_model.save_pretrained("Qwen3-14B-Abliterated-v2-nf4")
tokenizer.save_pretrained("Qwen3-14B-Abliterated-v2-nf4")
Limitations & Ethics
May amplify training data biases.
Not suitable for production without alignment.
For commercial use: review original licenses.
Contact
Open an issue or reach out to ikarius on Hugging Face.
Last updated: November 12, 2025
Original Model Credits-
Abliteration:huihui-ai
Support the project
Base Model:Qwen/Qwen3-14B