title: DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16
license: apache-2.0
language:
- en
pipeline_tag: text-generation
base_model: - DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
library_name: transformers
tags: - qwen3
- quantized
- w4a16
- auto-round
- gptq-compatible
- uncensored
- roleplay
- creative-writing
DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16
W4A16 quantized version of the DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU model, optimized for efficient inference on consumer hardware.
Quantization Details
| Parameter | Value |
|---|---|
| Method | AutoRound v0.12.3 |
| Scheme | W4A16 (4-bit int weights, 16-bit fp activations) |
| Group Size | 128 |
| Symmetric | Yes |
| Packing Format | auto_round:auto_gptq (GPTQ compatible) |
| Dataset | NeelNanda/pile-10k |
| Calibration Samples | 256 |
| Sequence Length | 4096 |
| Iterations | 512 |
| Seed | 42 |
Layers Kept at FP16
linear_attn.in_proj_aandlinear_attn.in_proj_b(all layers)embed_tokens- All
visual.*layers - All
mtp.*(multi-token prediction) layers lm_head
Model Card
- Architecture: Qwen3.8-27B (Qwen3_5ForConditionalGeneration)
- Parameters: 27B
- Hidden Size: 5120
- Layers: 64 (hybrid linear + full attention)
- Attention Heads: 24 (4 KV heads)
- Context Length: 262,144 tokens
- Vision: Yes (Qwen3.5 Vision encoder)
- Multimodal: Supports text, images, and video
Usage
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16",
device_map="auto",
torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
"DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16"
)
messages = [
{"role": "user", "content": "Write a short poem about quantum entanglement."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
llama.cpp / llama-server
The model is GPTQ-compatible. Convert to GGUF for llama.cpp inference:
python convert_hf_to_gguf.py . --outtype q4_K_M
Hardware Requirements
| VRAM | Notes |
|---|---|
| ~14 GB | Minimum with KV cache optimization |
| 16 GB | Comfortable for most use cases |
| 24 GB+ | Full context, longer generations |
Generation Parameters
Recommended settings from generation_config.json:
- Temperature: 1.0
- Top-k: 20
- Top-p: 0.95
- Do sample: true
License
Apache-2.0
Citation
@misc{davidau2025qwen38,
title={DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU},
author={DavidAU},
year={2025},
howpublished={\url{https://huggingface.co/DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU}}
}