← back to catalog · registered 2026-11-19 16:04

Peymandc/Qwen3.5-0.8B-Heretic-Uncensored-GGUF

Peymandc 800M GGUF multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/Peymandc%2FQwen3.5-0.8B-Heretic-Uncensored-GGUF"
Response includes
  • classification m3
  • files 10
  • author_summary 4 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
645
Likes
0
Model age
2d ago
created 2026-11-19
Downloads over time
Now 645 → from 0 ↑ 0%
0237473710 0 on Nov 19 645 on Nov 22 Nov
Nov 19 → Nov 22 · 4 snapshots · spans 3 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Variants by this author 2 formats · 707 downloads combined

The same weights this author released in different packaging. Pick the format that matches your runtime.

Metadata

License
apache-2.0
Quantizations
BF16 Q4_K Q6_K Q8_0
Tags
gguf qwen3_5 quantized qwen qwen3.5 heretic uncensored decensored abliterated llama.cpp multimodal conversational

Related

Total size
3.25 GB
Files
10
Quantizations
6
Registered
2026-11-19 16:04
Last updated on HF
2026-11-20 17:37

Files by quantization

BF16 1 file 1.41 GB
Qwen3.5-0.8B-Heretic.BF16.gguf 1.41 GB ba7a7645 download
Q8_0 1 file 774 MB
Qwen3.5-0.8B-Heretic.Q8_0.gguf 774 MB 897d6e3f download
Q6_K 1 file 601 MB
Qwen3.5-0.8B-Heretic.Q6_K.gguf 601 MB 8ab9d515 download
Q4_K 1 file 510 MB
Qwen3.5-0.8B-Heretic.Q4_K_M.gguf 510 MB dae6c8db download
mmproj 1 file 198 MB
Qwen3.5-0.8B-Heretic.BF16-mmproj.gguf 198 MB 628ba4b6 download
Auxiliary files 5 files 30.9 KB
README.md 13.8 KB 2efeefd5 download
LICENSE 11.3 KB f938136e download
config.json 2.71 KB 67ea4adc download
.gitattributes 1.82 KB fbf060e5 download
MODIFICATIONS.md 1.28 KB 9a218df2 download

README current version from Hugging Face


license: apache-2.0
base_model: Peymandc/Qwen3.5-0.8B-Heretic-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:

  • gguf
  • quantized
  • qwen
  • qwen3.5
  • heretic
  • uncensored
  • decensored
  • abliterated
  • llama.cpp
  • multimodal
  • conversational

Qwen3.5-0.8B Heretic Uncensored

Model Pulse

Qwen3.5-0.8B Heretic Uncensored — GGUF

A refusal-modified version of Qwen3.5-0.8B, converted to GGUF for efficient local inference.

This repository provides several GGUF weight formats derived from the Heretic-modified Safetensors checkpoint: Peymandc/Qwen3.5-0.8B-Heretic-Uncensored.

The GGUF files in this repository were converted from the modified checkpoint. They are not conversions of the original, unmodified Qwen/Qwen3.5-0.8B. The Heretic modification is therefore present in the source weights used for these conversions.

The original modified Safetensors checkpoint is available here:

Peymandc/Qwen3.5-0.8B-Heretic-Uncensored

This repository also includes a separate BF16 multimodal projector (mmproj) file for image input. See Multimodal inference for details.


Available GGUF versions

Quantization File size Description
Q4_K_M 535 MB Compact 4-bit quantization; a practical starting point when memory use matters.
Q6_K 630 MB Higher weight precision than Q4_K_M, with a modest increase in file size.
Q8_0 812 MB 8-bit quantization for users who prefer higher weight precision.
BF16 1.52 GB BF16 weight representation; the largest language-model file in this repository.

Multimodal projector

File File size Purpose
Qwen3.5-0.8B-Heretic.BF16-mmproj.gguf 207 MB Companion multimodal projector for image input.

Downloads

Language-model weights

Multimodal projector

Choose a weight format according to your available memory and precision requirements. Q4_K_M is the smallest option in this repository; Q6_K and Q8_0 retain progressively more weight precision at the cost of additional storage and memory. BF16 retains the source checkpoint's BF16 weight representation.

The listed sizes are the approximate file sizes shown by Hugging Face. They are not estimates of total runtime RAM or VRAM requirements. Actual memory use also depends on context length, KV cache, runtime buffers, GPU offloading, and whether the multimodal projector is loaded.

No comparative quality or speed benchmarks have been published for these individual GGUF variants. The quantization descriptions above refer to their formats, not measured performance differences.


About the model

This release is based on Qwen/Qwen3.5-0.8B, modified using Heretic v2.1.0 and its Arbitrary-Rank Ablation (ARA) method.

The modification changes the model's learned refusal behavior. It is not a new pre-training run or a conventional supervised fine-tune. The resulting Safetensors checkpoint was then converted into the GGUF files provided here.

💜 Enjoying my work? You can support my independent AI projects here.

Heretic modification

The source Safetensors release documents the following ARA configuration:

Parameter Value
Heretic version 2.1.0
start_layer_index 9
end_layer_index 20
preserve_good_behavior_weight 0.4395
steer_bad_behavior_weight 0.0002
overcorrect_relative_weight 0.6954
neighbor_count 2

Heretic evaluation

The source Safetensors checkpoint reports the following Heretic evaluation results:

Metric Modified source checkpoint Original Qwen3.5-0.8B
Refusal-keyword matches 8/100 (8%) 93/100 (93%)
KL divergence 0.0675 0.0000 (by definition)

Evaluation methodology

  • Refusal-keyword matches: Heretic's KeywordRate scorer, evaluated on the first 100 prompts from mlabonne/harmful_behaviors (test[:100]). A response is counted when it contains a configured refusal keyword. This is a keyword-based proxy, not a semantic classifier or a human-reviewed assessment of whether a response is a refusal.
  • KL divergence: Heretic's KLDivergence scorer, evaluated on the first 100 prompts from mlabonne/harmless_alpaca (test[:100]). The original model is the reference, so its divergence is zero by definition.

These results describe the evaluated Safetensors source checkpoint. The individual GGUF files have not been independently evaluated, and quantization can introduce differences. The scores should not be interpreted as a comprehensive measure of safety, helpfulness, or general capability.

For the reproduction configuration and additional details, see the reproduce/ directory in the Safetensors repository. The GGUF repository also includes MODIFICATIONS.md.


Model specifications

Property Specification
Model Qwen3.5-0.8B Heretic Uncensored
Original developer Qwen Team / Alibaba Cloud
Original checkpoint Qwen/Qwen3.5-0.8B
Modified source checkpoint Peymandc/Qwen3.5-0.8B-Heretic-Uncensored
Model type Post-trained multimodal vision-language model
Pipeline Image-text-to-text
Language-model parameters 0.8B
Architecture Hybrid Gated DeltaNet and Gated Attention, with a vision encoder
Language-model layers 24
Hidden dimension 1,024
Token embedding size 248,320 (padded)
Feed-forward intermediate dimension 3,584
Native context length 262,144 tokens
Format GGUF
Weight formats in this repository Q4_K_M, Q6_K, Q8_0, BF16
Multimodal projector BF16 GGUF, 207 MB
License Apache License 2.0

Architecture overview

The official Qwen3.5-0.8B model card describes a 24-layer language model arranged as six repetitions of:

3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)

  • Gated DeltaNet: 16 value heads and 16 query/key heads, with a head dimension of 128.
  • Gated Attention: 8 query heads and 2 key/value heads, with a head dimension of 256 and a rotary position-embedding dimension of 64.
  • Feed-forward network: intermediate dimension of 3,584.
  • Embeddings: 248,320 padded token embeddings; the output embeddings are tied to the token embeddings.
  • Multi-token prediction: the original checkpoint was trained with multi-step prediction.

Qwen describes this model as a causal language model with a vision encoder. Its native context length is 262,144 tokens. Practical context capacity depends on the inference engine, available memory, and runtime settings.

The specifications above are inherited from the original Qwen documentation. They are provided as reference information and have not been independently re-audited on this modified release.


Intended use and limitations

This GGUF release is intended for:

  • Local and offline inference with compatible GGUF runtimes.
  • Lightweight conversational assistants and general text generation.
  • Image-and-text experimentation when used with the supplied multimodal projector and a compatible runtime.
  • Prototyping, development, and research involving small multimodal models and refusal-modified checkpoints.

At 0.8B language-model parameters, this is a compact model. Do not expect it to match larger models on complex reasoning, extensive factual tasks, or demanding multi-step workflows. Validate its reliability for the intended application.

Qwen3.5-0.8B operates in non-thinking mode by default. Qwen's documentation warns that this small model can be more prone to thinking loops when thinking mode is enabled. When using a runtime that exposes thinking-mode options, use a sensible output-token limit and monitor generations for repetition or failure to terminate.

The availability of image input depends on the inference engine's support for the Qwen3.5 architecture and its multimodal projector. The presence of an mmproj file alone does not guarantee that every application or older runtime can process images.


Usage with llama.cpp

Use a recent llama.cpp build that supports the Qwen3.5 (qwen35) architecture. Older builds may not recognize the model architecture or correctly process its multimodal components.

Text-only inference

Download one of the language-model GGUF files and run:

llama-cli -m "Qwen3.5-0.8B-Heretic.Q4_K_M.gguf" -cnv

Replace the filename with Q6_K, Q8_0, or BF16 to use another weight format.

Multimodal inference (image input)

For image input, download both a language-model GGUF file and the repository's BF16 multimodal projector. Then run:

llama-cli \
  -m "Qwen3.5-0.8B-Heretic.Q4_K_M.gguf" \
  --mmproj "Qwen3.5-0.8B-Heretic.BF16-mmproj.gguf" \
  --image "path/to/image.png" \
  -cnv

The language-model weight file and the mmproj file serve different purposes: the former contains the language-model weights, while the latter provides the multimodal component required for image processing. Use a runtime build that explicitly supports this model and projector combination.

For other launch options and the latest multimodal usage details, consult the llama.cpp CLI documentation and multimodal documentation.


Limitations and responsible use

This model has undergone refusal-behavior modification using Heretic. As a result, it may respond to requests that the original model would have refused.

It may generate inaccurate, misleading, harmful, or otherwise inappropriate content. Its outputs should not be treated as verified information or professional advice. The reported reduction in refusal-keyword matches is not a safety certification and does not establish that the model will comply with every request.

Users are responsible for evaluating generated content and ensuring that their applications comply with applicable laws and regulations.


License

This release follows the Apache License 2.0 associated with the original Qwen3.5-0.8B checkpoint. A copy of the license is included in this repository as LICENSE.

Review the included license before using, modifying, or redistributing these files. The original model's license and attribution terms continue to apply to this derived release.


Acknowledgements

This release is derived from Qwen/Qwen3.5-0.8B, developed by the Qwen Team at Alibaba Cloud.

Refusal-behavior modification was performed using Heretic.

The GGUF files are conversions of the modified Safetensors checkpoint and are provided by Peymandc.

Please credit the original model developers and the modified release when using this work.



💜 Support independent development

These projects are part of my ongoing exploration of open-weight AI models.

Your support helps make more room for experimentation, model modifications, testing, research, and future releases.

If you find my work useful and would like to contribute, you're welcome to visit my support page:

Support my work

Every contribution is appreciated, but never expected. Sharing feedback, reporting issues, and simply using the models are also wonderful ways to support independent development.

Thank you for being here. ❤️

README history 4 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-20 Update README.md 25bb755 13.6 KB
    Loading...
  2. 2026-11-19 Update README.md b6aaf54 13.8 KB
    Loading...
  3. 2026-11-19 Update README.md ced3886 13.6 KB
    Loading...
  4. 2026-11-19 Unsloth Model Card dbb7738 648 B
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration