license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
base_model: DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
base_model_relation: quantized
tags: - qwen3_5
- qwen3_8
- heretic
- uncensored
- finetune
- Cold Fusion
- GAIN Training
- Multi-stage tuning
- all use cases
- unsloth
- conversational
- compressed-tensors
- safetensors
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU — W4A16 AutoRound
This repository is a quantized version of:
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
Original model:
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
The underlying model weights are not a new fine-tune or merge. This repository contains a W4A16 quantization of the original DavidAU model, prepared for efficient inference on consumer GPUs.
Quantization
- Weight format: W4A16
- Weight precision: INT4
- Activation precision: BF16 / FP16-compatible 16-bit activations
- Quantization strategy: symmetric, group-wise
- Group size: 128
- Quantization format: llm_compressor
- Packed format: pack-quantized
- Algorithm: SignRound
- Quantization tool: AutoRound 0.15.0
- Calibration dataset: NeelNanda/pile-10k
- Calibration samples: 128
- Optimization iterations: 200
- Sequence length: 2048
The 64 transformer language layers were quantized to W4A16 INT4 group-128.
Head / Embedding / MTP preparation
For compatibility with the target runtime, the following components were additionally requantized using the prepared INT8 group-128 representation:
lm_headmodel.language_model.embed_tokens- Qwen MTP module linears
The MTP draft vocabulary was also prepared for speculative decoding.
Runtime compatibility
This checkpoint was prepared and tested with the
syv-ai/qwen38-27b-rtx3090
runtime stack using:
- vLLM 0.27.1
- Compressed-Tensors
- Marlin W4A16
- FlashAttention 2
- Qwen GDN/FLA kernels
- MTP speculative decoding
- KVarN KV-cache support for long-context configurations
Performance
On a single RTX 3090 Ti, using the syv-ai single-user MTP configuration with 4 speculative tokens, this quantized checkpoint reached approximately:
~110 tokens/s decode
The measured result is runtime/configuration dependent and should not be treated as a universal benchmark.
Attribution
Original model: DavidAU
Quantization/preparation: Punal Manalan
This repository is intended to preserve the behavior and identity of the original model while providing a significantly smaller and faster-to-run W4A16 representation.
License
This repository follows the license of the original model:
Apache-2.0