Quantization made by Richard Erkhov.
SmolLM2-1.7B-Instruct-abliterated - AWQ
- Model creator: https://huggingface.co/huihui-ai/
- Original model: https://huggingface.co/huihui-ai/SmolLM2-1.7B-Instruct-abliterated/
Original model description:
library_name: transformers
license: apache-2.0
language:
- en
tags: - Safetensors
- conversational
- text-generation-inference
- abliterated
- uncensored
base_model: - HuggingFaceTB/SmolLM2-1.7B-Instruct
huihui-ai/SmolLM2-1.7B-Instruct-abliterated
This is an uncensored version of HuggingFaceTB/SmolLM2-1.7B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
If the desired result is not achieved, you can clear the conversation and try again.
How to use
Transformers
pip install transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "huihui-ai/SmolLM2-1.7B-Instruct-abliterated"
device = "cuda" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
# for multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
messages = [{"role": "user", "content": "What is the capital of France."}]
input_text=tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0]))