license: other
base_model: NousResearch/Hermes-3-Llama-3.1-8B
tags:
- text-generation-inference
- llama-cpp
- gguf
- heretic
pipeline_tag: text-generation
quantized_by: llama.cpp
L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8B-GGUF-heretic
This repository contains GGUF format quantizations of the model using llama.cpp.
3/100 Refusal via Heretic 1.4 by https://github.com/p-e-w/heretic
Available Quants:
- Q4_K_M: 4-bit medium quantization (balanced performance/size).
- Q5_K_M: 5-bit medium quantization (recommended sweet spot for 8B).
- Q6_K: 6-bit quantization (extremely close to unquantized quality).
- Q8_0: 8-bit standard quantization.
How to use with llama.cpp:
./llama-cli -m L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8B-heretic.Q5_K_M.gguf -n 512 -ngl 999 -co -i