Lily-Cybersecurity-7B-Uncensored-GGUF_Q5_K_M
Credits & License
This is a re-quantized version of an existing GGUF model. (And size of it converted from 13.5GB to 4.07GB)
So it can be used on CPU with 16GB of RAM (Also can be run on Google Colab).
Lineage:
- Original Model: segolilylabs/Lily-Cybersecurity-7B-v0.2 (License: Apache 2.0)
- Intermediate GGUF (source for re-quantization): NiroshanDb23/Lily-Cybersecurity-7B-Uncensored-GGUF (quantized_by: NiroshanDH, includes abliteration modifications)
My Contribution:
- Re-quantized the intermediate GGUF file into Q4_K_M and Q5_K_M formats to reduce RAM usage for inference.
- Alarm: Quantized version can hallucinate.
- Recommended context window: 4096 tokens (or higher if hardware permits).
License:
This derivative work is distributed under the Apache 2.0 license, inherited from the original model.
Please refer to the original model card for full terms and disclaimer.