tags:
- GPTQ
- qwen3_5
- mtp
- text-generation-inference
- llm-compressor
- vllm
- stable
- abliterated
- nvfp4
- nvidia
- speculative-decoding
- nvidia-blackwell
- rtx-pro-6000
- SM120
- GB202
- B200
- B300
- FP4
- blackwell
- Qwen3.5
- Qwen3.6
- Qwen3.8
- uncensored
- decensored
- imatrix
license: apache-2.0
base_model: - prithivMLmods/Qwen3.5-9B-DS-v4-Flash-uncensored-heretic
language: - en
pipeline_tag: image-text-to-text
library_name: transformers

Qwen3.5-9B-DS-v4-Flash-uncensored-heretic-NVFP4
Qwen3.5-9B-DS-v4-Flash-uncensored-heretic-NVFP4 is an NVFP4-quantized version of Qwen3.5-9B-DS-v4-Flash-uncensored-heretic, quantized using GPTQ (Accurate Post-Training Quantization for Generative Pre-trained Transformers). GPTQ calibration plays a critical role in stabilizing and optimizing NVFP4 compression, helping preserve accuracy close to the original weights while significantly reducing memory requirements and improving inference efficiency on NVFP4-compatible hardware.
[!NOTE]
MTP — Speculative Decoding: Yes | This is an experimental quantized release and may exhibit unexpected behaviors or reasoning artifacts in certain scenarios. Outputs should be independently evaluated.
Quantization Recipe
The model is quantized using the GPTQModifier with the NVFP4 quantization scheme.
default_stage:
default_modifiers:
GPTQModifier:
targets: [Linear]
ignore: [lm_head, "re:.*visual.*"]
scheme: NVFP4
weight_observer: imatrix_mse
bypass_divisibility_checks: false
requires_calibration_data: true
block_size: 128
dampening_frac: 0.01
actorder: static
batched_quantization: auto
Model Files
| File Name | File Type | File Size | File Link | Description |
|---|---|---|---|---|
| chat_template.jinja | Jinja Template | 7.76 kB | Link | Chat template for formatting conversations and prompts. |
| config.json | JSON | 9.47 kB | Link | Core model architecture and configuration parameters. |
| generation_config.json | JSON | 153 Bytes | Link | Default text generation parameters (e.g., temperature, top_p). |
| model-00001-of-00002.safetensors | Safetensors | 5.18 GB | Link | Model weight shard 1/2. Required to load the model. |
| model-00002-of-00002.safetensors | Safetensors | 3.7 GB | Link | Model weight shard 2/2. Required to load the model. |
| model.safetensors.index.json | JSON | 153 kB | Link | Index file mapping model weights to their respective shards. |
| model_mtp.safetensors | Safetensors | 487 MB | Link | Multi-Token Prediction (MTP) auxiliary model weights, used to speed up generation. |
| processor_config.json | JSON | 1.33 kB | Link | Configuration for the multimodal processor. |
| recipe.yaml | YAML | 334 Bytes | Link | Configuration file detailing the quantization or training recipe (e.g., NVFP4 parameters). |
| tokenizer.json | JSON | 20 MB | Link | Complete tokenizer vocabulary and encoding rules. |
| tokenizer_config.json | JSON | 1.33 kB | Link | Tokenizer-specific configuration parameters. |
License and Attribution
This model is based on and/or incorporates the following open-source projects and models:
- Qwen3.5-9B (Base): https://huggingface.co/Qwen/Qwen3.5-9B
- DeepSeek V4 Flash: Reasoning traces used for reasoning-oriented fine-tuning.
- llm-compressor: https://github.com/vllm-project/llm-compressor
This model is released under the Apache License 2.0.