license: apache-2.0
language:
- en
base_model: huihui-ai/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated
tags: - qwen3.5
- vision
- text-generation
- image-text-to-text
- reasoning
- uncensored
- long-context
- 1M-context
- abliterated
- gguf
- llama.cpp
Abiray/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-GGUF
This repository contains GGUF format quantizations of huihui-ai/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated, which is an uncensored version of empero-ai's Qwythos-9B-Claude-Mythos-5-1M model stripped of its refusal vectors via abliteration techniques.
Additionally, this repository includes the necessary multimodal projection files (mmproj) required to utilize the model's vision/image-to-text capabilities locally.
Provided Files & Quantization Details
| File Name | Size | Description |
|---|---|---|
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q3_K_L.gguf |
5.05 GB | 3-bit ultra-light quantization, long variant. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q3_K_M.gguf |
4.74 GB | 3-bit medium quantization. Lowest resource footprint. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q4_K_M.gguf |
5.78 GB | 4-bit medium quantization. Recommended balance of speed and quality. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q4_K_S.gguf |
5.49 GB | 4-bit small variant footprint. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q5_K_M.gguf |
6.64 GB | 5-bit medium quantization. High quality preservation. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q6_K.gguf |
7.56 GB | 6-bit quantization. Extremely near-lossless performance. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q8_0.gguf |
9.79 GB | 8-bit standard quantization. Maximum precision text execution. |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.mmproj-Q8_0.gguf |
624 MB | Quantized vision projector adapter (required for vision capabilities). |
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.mmproj-f16.gguf |
918 MB | Unquantized 16-bit float vision projector adapter for absolute vision fidelity. |
Deployment & Usage Instructions
Ensure you are utilizing a recent build of llama.cpp to natively handle modern Qwen 3.5 architectures and context window stretching constraints.
1. Pure Text Mode (CLI Inference)
To interact with the model purely over terminal text-generation pipelines:
./llama-cli \
-m Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated.Q4_K_M.gguf \
-ngl 99 \
-c 32768 \
-fa on \
-p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite a short story about an AI break-out.<|im_end|>\n<|im_start|>assistant\n"