license: apache-2.0
language:
- en
- zh
base_model: - Qwen/Qwen3.5-35B-A3B
- HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive
pipeline_tag: image-text-to-text
library_name: transformers
tags: - qwen3_5
- qwen35
- moe
- vllm
- qwen
- conversational
- quantized
- compressed
- compressed-tensors
- llm-compressor
- fp8
- qwen3_5_moe
- uncensored
- unfiltered
- ablation
- optimized
Qwen3.5-35B-A3B Uncensored (FP8_BLOCK)
A safetensors conversion and quantization of HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive (GGUF).
Model Details
| Architecture | Qwen3.5 MoE hybrid attention (30 GDN + 10 full standard attention layers) |
| Parameters | 35B-A3B |
| Base model | Qwen/Qwen3.5-35B-A3B |
| Source GGUF | HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive |
| Format | BF16 (Mixed/Compressed) |
| Quantization | FP8_BLOCK applied to Linear transformer layers. |
| Stripped layers | Multi-Token Prediction (MTP) due to original HF -> GGUF conversion. |
| Conversion type | Lossless GGUF to safetensors conversion + merge with base model vision layers + Block-wise quantization |
| Unquantized weights | Coming Soon |
Conversion Details
Converted using coming soon, which reverses transforms applied during HF -> GGUF conversion.
The vision encoder weights are copied directly from the official Qwen/Qwen3.5-35B-A3B base model, after confirming the vision encoder (mmproj) was not modified in the source GGUF.
Next, the linear weights of the transformer blocks were quantized to F8_E4M3 using llm-compressor.
Test Inference Details
- 1x A100 (80GB)
- Python 3.12
- vllm & transformers version:
transformers5.5.0vllmnightly (latest commit tested: 7b743ba)
- vLLM online serve flags:
--quantization compressed-tensors--max-model-len 16384--gpu-memory-utilization 0.9140withVLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=1environmental variable--limit-mm-per-prompt.image 4--enable-prefix-caching--enable-expert-parallel--reasoning-parser qwen3--default-chat-template-kwargs {"enable_thinking": false}disabled thinking/reasoning forvllm>=0.18.1- Note: Used for batch image captioning tests.
Credits
- Base model: Qwen Team — Qwen3.5
- Ablation/uncensoring: HauhauCS
- Quantization tool: llm-compressor