license: apache-2.0
base_model:
- DavidAU/Qwen3.8-27B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF
Qwen3.8-27B-Ultimate-Heretic-Uncensored (Ninfer v3 Format)
This is the Ninfer v3 converted version of the DavidAU/Qwen3.8-27B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF model.
I have just done the conversion (NInferEZ Converter) of Q4_K_M GGUF to Ninfer v3 with included vision model and MTP support.
🚀 Testing & Benchmark Setup
Tested with Ninfer-all on Windows 10 with full context size (262,144 tokens) on an RTX 3090.
Inference Command (Windows)
Run directly in CMD or PowerShell:
"Z:\ninfer-windows-x64-cuda13.4-sm75-80-86-89-120a-6d8dc3c1e0e1\ninfer-serve.exe" "Z:\Models\Qwen3.8-27B-Ultimate-Heretic-Uncn-Turbo-Brilli-50x-AGI-NEO-MTP-Q4_K_M-Ninfer-v3.ninfer" --host 0.0.0.0 --port 1234 --max-concurrency 3 --max-context 262144 --kv-capacity 262144 --kv-dtype rk4v4-e8 --spec mtp --draft-tokens 3 --embedding-q4 --gdn-state-fp16 --prefill-cublas --prefill-chunk 4096 --max-pending-requests 16 --pending-timeout-ms 600000 --vision --vision-residency overlay --max-private-continuations 8 --max-shared-prefixes 8 --host-state-slots 32 --host-kv-mib 8192 --auto-prefix-grid --model-id Qwen38Dense
⚠️ Thermal Note & Power Limitation (HP Z640)
I have received ~90 tps on the RTX 3090, but with restricted power of the GPU to 250W:
nvidia-smi -pl 250
This is necessary because the GPU Memory Junction Temperature goes over 100°C on an unmodified HP Z640 case with higher GPU power.
Recommendation: I advise you to check the temperature of the GPU memory with every model change (I use HWiNFO for checking and MSI Afterburner to increase fan speed).
🤝 Credits
All credits go to all people for their respective work!