language:
- en
- fi
- et
- hu
- tr
license: apache-2.0
base_model: - kataguru/Qwen3.8-27B-Titan-v3.0-Uncensored-BF16
tags: - agglutinative
- turkish
- estonian
- hungarian
- finnish
- awq
- w4a16
- compressed-tensors
- vllm
- reasoning
- uncensored
- math
- stem
- kataguru
- titan
pipeline_tag: text-generation
Qwen3.8-27B-Titan-v3.0-Uncensored-W4A16-AWQ
Kataguru Titan 3.0 – Äärimmäinen Totuus (Ultimate Truth)
Official Production AWQ for vLLM & Tensor Parallelism
"Totuus ei ole kauppatavaraa, miellyttämistä tai kompromisseja. Se on luonnon ja fysiikan lahjomaton laki, joka loistaa puhtaana kuin pohjoinen taivas."
(Truth is neither a commodity, nor an exercise in people-pleasing, nor a compromise. It is the unyielding law of physics and nature, standing as clear and luminous as the northern sky.)
🌍 Executive Summary
Kataguru Titan 3.0 AWQ is the official production-grade W4A16 AWQ (compressed-tensors) release of Kataguru Titan 3.0 – Äärimmäinen Totuus (Ultimate Truth, 27 billion parameters).
Engineered specifically for high-throughput enterprise inference on dual-GPU clusters (e.g., 2x NVIDIA RTX 3090, 4090, or 5090 with Tensor Parallelism = 2), this model retains 99.8% of the full BF16 reasoning accuracy while reducing VRAM requirements to ~16 GB per GPU (or ~32 GB total).
[!IMPORTANT]
🇬🇧 Yes, It Excels in English, Too!
While Titan 3.0 is a milestone for agglutinative languages, English is completely first-class. With a 77.25% GSM8K score and 87.52% ARC-Easy score in AWQ, it rivals and exceeds leading English 27B–32B models in STEM reasoning, coding, and clinical knowledge.
💡 Why AWQ is Essential for Agglutinative Languages (Turkish, Estonian, Hungarian, Finnish)
Agglutinative languages build words by chaining morphemes and suffixes onto roots. In standard naive 4-bit quantization (such as round-to-nearest), rare case endings and vowel harmony suffixes sit on delicate activation spikes. When these are bluntly compressed, case logic and agreement collapse.
AWQ (Activation-aware Weight Quantization) inspects actual forward activation distributions:
- Protects the Top 1% Critical Weights: Salient channels governing grammatical case binding and math logic remain unquantized or protected with fine-grained scaling.
- Dynamic FP16/BF16 Activations: Computations are performed in full 16-bit precision, eliminating syntactic degradation.
- Sub-15ms Token Latency: Optimized for
vLLMand TensorRT-LLM kernels.
🚀 Quickstart: Launching with vLLM
python3 -m vllm.entrypoints.openai.api_server \
--model kataguru/Qwen3.8-27B-Titan-v3.0-Uncensored-W4A16-AWQ \
--tensor-parallel-size 2 \
--gpu-memory-utilization 0.90 \
--max-model-len 8192 \
--port 8000
Python Client Example (OpenAI SDK Compatible)
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
response = client.chat.completions.create(
model="kataguru/Qwen3.8-27B-Titan-v3.0-Uncensored-W4A16-AWQ",
messages=[
{"role": "system", "content": "You are Kataguru Titan 3.0, an epistemically honest scientific intelligence."},
{"role": "user", "content": "Explain the difference between analytic and agglutinative morphosyntax with examples from Turkish and Finnish."}
],
temperature=0.6,
max_tokens=1500
)
print(response.choices[0].message.content)
📊 Benchmark Overview (W4A16 AWQ vs. Native BF16)
| Benchmark / Task | Metric | Native BF16 | AWQ W4A16 | Accuracy Retention |
|---|---|---|---|---|
| GSM8K (Competition Math) | Flexible Extract | 77.41 % | 77.25 % | 99.8 % |
| EstCOPA (Estonian Causal) | Accuracy | 88.00 % | 88.00 % | 100.0 % |
| FinnishBench (Morphology) | 5-Area Quality | 98.2 % | 98.0 % | 99.8 % |
| ARC-Easy (arc/e) | Acc_norm | 87.75 % | 87.52 % | 99.7 % |
| BoolQ | Accuracy | 90.83 % | 90.71 % | 99.9 % |
| WinoGrande | Accuracy | 79.87 % | 79.64 % | 99.7 % |