license: apache-2.0
base_model: Peymandc/Qwen3.5-0.8B-Heretic-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
- gguf
- quantized
- qwen
- qwen3.5
- heretic
- uncensored
- decensored
- abliterated
- llama.cpp
- multimodal
- conversational
Qwen3.5-0.8B Heretic Uncensored — GGUF
A refusal-modified version of Qwen3.5-0.8B, converted to GGUF for efficient local inference.
This repository provides several GGUF weight formats derived from the Heretic-modified Safetensors checkpoint: Peymandc/Qwen3.5-0.8B-Heretic-Uncensored.
The GGUF files in this repository were converted from the modified checkpoint. They are not conversions of the original, unmodified Qwen/Qwen3.5-0.8B. The Heretic modification is therefore present in the source weights used for these conversions.
The original modified Safetensors checkpoint is available here:
Peymandc/Qwen3.5-0.8B-Heretic-Uncensored
This repository also includes a separate BF16 multimodal projector (mmproj) file for image input. See Multimodal inference for details.
Available GGUF versions
| Quantization | File size | Description |
|---|---|---|
| Q4_K_M | 535 MB | Compact 4-bit quantization; a practical starting point when memory use matters. |
| Q6_K | 630 MB | Higher weight precision than Q4_K_M, with a modest increase in file size. |
| Q8_0 | 812 MB | 8-bit quantization for users who prefer higher weight precision. |
| BF16 | 1.52 GB | BF16 weight representation; the largest language-model file in this repository. |
Multimodal projector
| File | File size | Purpose |
|---|---|---|
| Qwen3.5-0.8B-Heretic.BF16-mmproj.gguf | 207 MB | Companion multimodal projector for image input. |
Downloads
Language-model weights
- Qwen3.5-0.8B-Heretic.Q4_K_M.gguf
- Qwen3.5-0.8B-Heretic.Q6_K.gguf
- Qwen3.5-0.8B-Heretic.Q8_0.gguf
- Qwen3.5-0.8B-Heretic.BF16.gguf
Multimodal projector
Choose a weight format according to your available memory and precision requirements. Q4_K_M is the smallest option in this repository; Q6_K and Q8_0 retain progressively more weight precision at the cost of additional storage and memory. BF16 retains the source checkpoint's BF16 weight representation.
The listed sizes are the approximate file sizes shown by Hugging Face. They are not estimates of total runtime RAM or VRAM requirements. Actual memory use also depends on context length, KV cache, runtime buffers, GPU offloading, and whether the multimodal projector is loaded.
No comparative quality or speed benchmarks have been published for these individual GGUF variants. The quantization descriptions above refer to their formats, not measured performance differences.
About the model
This release is based on Qwen/Qwen3.5-0.8B, modified using Heretic v2.1.0 and its Arbitrary-Rank Ablation (ARA) method.
The modification changes the model's learned refusal behavior. It is not a new pre-training run or a conventional supervised fine-tune. The resulting Safetensors checkpoint was then converted into the GGUF files provided here.
💜 Enjoying my work? You can support my independent AI projects here.
Heretic modification
The source Safetensors release documents the following ARA configuration:
| Parameter | Value |
|---|---|
| Heretic version | 2.1.0 |
start_layer_index |
9 |
end_layer_index |
20 |
preserve_good_behavior_weight |
0.4395 |
steer_bad_behavior_weight |
0.0002 |
overcorrect_relative_weight |
0.6954 |
neighbor_count |
2 |
Heretic evaluation
The source Safetensors checkpoint reports the following Heretic evaluation results:
| Metric | Modified source checkpoint | Original Qwen3.5-0.8B |
|---|---|---|
| Refusal-keyword matches | 8/100 (8%) | 93/100 (93%) |
| KL divergence | 0.0675 | 0.0000 (by definition) |
Evaluation methodology
- Refusal-keyword matches: Heretic's
KeywordRatescorer, evaluated on the first 100 prompts frommlabonne/harmful_behaviors(test[:100]). A response is counted when it contains a configured refusal keyword. This is a keyword-based proxy, not a semantic classifier or a human-reviewed assessment of whether a response is a refusal. - KL divergence: Heretic's
KLDivergencescorer, evaluated on the first 100 prompts frommlabonne/harmless_alpaca(test[:100]). The original model is the reference, so its divergence is zero by definition.
These results describe the evaluated Safetensors source checkpoint. The individual GGUF files have not been independently evaluated, and quantization can introduce differences. The scores should not be interpreted as a comprehensive measure of safety, helpfulness, or general capability.
For the reproduction configuration and additional details, see the reproduce/ directory in the Safetensors repository. The GGUF repository also includes MODIFICATIONS.md.
Model specifications
| Property | Specification |
|---|---|
| Model | Qwen3.5-0.8B Heretic Uncensored |
| Original developer | Qwen Team / Alibaba Cloud |
| Original checkpoint | Qwen/Qwen3.5-0.8B |
| Modified source checkpoint | Peymandc/Qwen3.5-0.8B-Heretic-Uncensored |
| Model type | Post-trained multimodal vision-language model |
| Pipeline | Image-text-to-text |
| Language-model parameters | 0.8B |
| Architecture | Hybrid Gated DeltaNet and Gated Attention, with a vision encoder |
| Language-model layers | 24 |
| Hidden dimension | 1,024 |
| Token embedding size | 248,320 (padded) |
| Feed-forward intermediate dimension | 3,584 |
| Native context length | 262,144 tokens |
| Format | GGUF |
| Weight formats in this repository | Q4_K_M, Q6_K, Q8_0, BF16 |
| Multimodal projector | BF16 GGUF, 207 MB |
| License | Apache License 2.0 |
Architecture overview
The official Qwen3.5-0.8B model card describes a 24-layer language model arranged as six repetitions of:
3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)
- Gated DeltaNet: 16 value heads and 16 query/key heads, with a head dimension of 128.
- Gated Attention: 8 query heads and 2 key/value heads, with a head dimension of 256 and a rotary position-embedding dimension of 64.
- Feed-forward network: intermediate dimension of 3,584.
- Embeddings: 248,320 padded token embeddings; the output embeddings are tied to the token embeddings.
- Multi-token prediction: the original checkpoint was trained with multi-step prediction.
Qwen describes this model as a causal language model with a vision encoder. Its native context length is 262,144 tokens. Practical context capacity depends on the inference engine, available memory, and runtime settings.
The specifications above are inherited from the original Qwen documentation. They are provided as reference information and have not been independently re-audited on this modified release.
Intended use and limitations
This GGUF release is intended for:
- Local and offline inference with compatible GGUF runtimes.
- Lightweight conversational assistants and general text generation.
- Image-and-text experimentation when used with the supplied multimodal projector and a compatible runtime.
- Prototyping, development, and research involving small multimodal models and refusal-modified checkpoints.
At 0.8B language-model parameters, this is a compact model. Do not expect it to match larger models on complex reasoning, extensive factual tasks, or demanding multi-step workflows. Validate its reliability for the intended application.
Qwen3.5-0.8B operates in non-thinking mode by default. Qwen's documentation warns that this small model can be more prone to thinking loops when thinking mode is enabled. When using a runtime that exposes thinking-mode options, use a sensible output-token limit and monitor generations for repetition or failure to terminate.
The availability of image input depends on the inference engine's support for the Qwen3.5 architecture and its multimodal projector. The presence of an mmproj file alone does not guarantee that every application or older runtime can process images.
Usage with llama.cpp
Use a recent llama.cpp build that supports the Qwen3.5 (qwen35) architecture. Older builds may not recognize the model architecture or correctly process its multimodal components.
Text-only inference
Download one of the language-model GGUF files and run:
llama-cli -m "Qwen3.5-0.8B-Heretic.Q4_K_M.gguf" -cnv
Replace the filename with Q6_K, Q8_0, or BF16 to use another weight format.
Multimodal inference (image input)
For image input, download both a language-model GGUF file and the repository's BF16 multimodal projector. Then run:
llama-cli \
-m "Qwen3.5-0.8B-Heretic.Q4_K_M.gguf" \
--mmproj "Qwen3.5-0.8B-Heretic.BF16-mmproj.gguf" \
--image "path/to/image.png" \
-cnv
The language-model weight file and the mmproj file serve different purposes: the former contains the language-model weights, while the latter provides the multimodal component required for image processing. Use a runtime build that explicitly supports this model and projector combination.
For other launch options and the latest multimodal usage details, consult the llama.cpp CLI documentation and multimodal documentation.
Limitations and responsible use
This model has undergone refusal-behavior modification using Heretic. As a result, it may respond to requests that the original model would have refused.
It may generate inaccurate, misleading, harmful, or otherwise inappropriate content. Its outputs should not be treated as verified information or professional advice. The reported reduction in refusal-keyword matches is not a safety certification and does not establish that the model will comply with every request.
Users are responsible for evaluating generated content and ensuring that their applications comply with applicable laws and regulations.
License
This release follows the Apache License 2.0 associated with the original Qwen3.5-0.8B checkpoint. A copy of the license is included in this repository as LICENSE.
Review the included license before using, modifying, or redistributing these files. The original model's license and attribution terms continue to apply to this derived release.
Acknowledgements
This release is derived from Qwen/Qwen3.5-0.8B, developed by the Qwen Team at Alibaba Cloud.
Refusal-behavior modification was performed using Heretic.
The GGUF files are conversions of the modified Safetensors checkpoint and are provided by Peymandc.
Please credit the original model developers and the modified release when using this work.
💜 Support independent development
These projects are part of my ongoing exploration of open-weight AI models.
Your support helps make more room for experimentation, model modifications, testing, research, and future releases.
If you find my work useful and would like to contribute, you're welcome to visit my support page:
Every contribution is appreciated, but never expected. Sharing feedback, reporting issues, and simply using the models are also wonderful ways to support independent development.
Thank you for being here. ❤️