Qwen2.5-1.5B-Abliterated — GGUF
GGUF conversion of AmareshHebbar/Qwen2.5-1.5B-Abliterated-latest,
quantized to Q4_K_M (5.08 BPW, 942 MiB / 986 MB).
The source model is distributed as bitsandbytes NF4 (double-quant) safetensors.
It was dequantized to F16 (bitsandbytes NF4 → F16), converted to GGUF with
llama.cpp, and quantized with llama-quantize … Q4_K_M.
Files
| file | size | notes |
|---|---|---|
Qwen2.5-1.5B-Abliterated-latest-Q4_K_M.gguf |
986 MB | --jinja ready; tokenizer.chat_template embedded; tokenizer.ggml.pre = qwen2 |
Usage (llama.cpp)
llama-server -m Qwen2.5-1.5B-Abliterated-latest-Q4_K_M.gguf --jinja -c 2048
Recommended sampling (from the source model's generation config, also embedded
as GGUF metadata): temperature=0.7, top_p=0.8, top_k=20, repeat_penalty=1.1.
Notes
- eos:
<|im_end|>(151645), pad:<|endoftext|>(151643) - architecture:
qwen2, context length 32768 (configurable at runtime) - Built for on-device inference with the PocketLLM Android app.