Jolex98/Qwen3.6-27B-uncensored-heretic-v2-GGUF

base_model: llmfan46/Qwen3.6-27B-uncensored-heretic-v2
license: apache-2.0
language:

• en
  library_name: gguf
  tags:
• gguf
• imatrix
• roleplay
• uncensored
• abliterated

About

This repository contains weighted/imatrix GGUF quantizations of llmfan46/Qwen3.6-27B-uncensored-heretic-v2.

Key Features of these Quants:

• Vision-Stripped (Text-Only): All vision-related tensors (mm_proj, visual, etc.) were stripped from the original .safetensors prior to conversion. This makes these GGUF files significantly lighter, faster to load, and perfectly optimized for text-based inference and Roleplay.
• Extreme Imatrix Calibration: The importance matrix (imatrix.dat) was calculated using a dedicated Roleplay dataset (calib_rp.txt). To ensure the absolute highest precision and retention of character reasoning and uncensored creativity, the calibration was run for an exhaustive ~8,000 chunks.
• Custom "Large" Variants: Included are handcrafted Q4_K_L and Q5_K_L variants, created by forcing the most sensitive components (Embeddings and Output tensors) to 8-bit precision (q8_0), bridging the gap between medium quants and Q6_K.

Provided Quants

(Files are sorted by size. For roleplay, the IQ4_K_M or Q4_K_L are highly recommended for a great balance of speed and intelligence).

DIY: How to create your own High-Quality Quants

For fellow adventurers wanting to quantize their own models at home, here is a mini-guide on how this specific workflow was executed using llama.cpp on Windows:

1. Strip Vision Tensors (For Text-Only Inference)
If the base model is multimodal, you save space by removing vision weights. Use a simple Python script with the safetensors library:

from safetensors.torch import load_file, save_file
# Load each .safetensors file, find keys containing "vision", "visual", "mm_proj", etc., delete them, and save.

2. Convert to FP16
Use the convert_hf_to_gguf.py script provided in the llama.cpp repository:

python llama.cpp/convert_hf_to_gguf.py ./base_model --outfile model_fp16.gguf --outtype f16

3. Calculate the Importance Matrix (Imatrix)
Download a solid calibration dataset (like calib_rp.txt for roleplay). Limit the GPU layers (-ngl) and CPU threads (-t) to prevent your system from crashing during this heavy process:

llama-imatrix.exe -m model_fp16.gguf -f calib_rp.txt -o imatrix.dat -c 512 --gpu-layers 12 -t 8

4. Final Quantization
Apply the imatrix to generate your desired formats:

# Standard
llama-quantize.exe --imatrix imatrix.dat model_fp16.gguf result_IQ4_K_M.gguf IQ4_K_M

# Custom Large (Q8_0 embeddings)
llama-quantize.exe --imatrix imatrix.dat --token-embedding-type q8_0 --output-tensor-type q8_0 model_fp16.gguf result_Q4_K_L.gguf Q4_K_M

A Personal Note

This repository is a personal project. It was my very first time diving into the world of LLM quantization, compiling GGUFs, and calculating importance matrices. I've put a lot of effort, time (and a couple of system crashes!) into making sure the process was done right and that the quality of these files is as high as possible. While I am just starting out and there's always room to learn, I hope these models serve you well.

Happy Roleplaying!