license: apache-2.0
base_model: TokenRhythm/NeoHorse-1-4B
library_name: gguf
pipeline_tag: text-generation
language: [en]
tags: [gguf, llama.cpp, 8gb, vram, uncensored, abliterated, heretic, agentic, tool-calling, imatrix, neohorse, qwen3.5, mmlu, gsm8k]
NeoHorse-1-4B · Sleipnir · 8 GB
Eight legs for eight gigabytes. An uncensored NeoHorse that still calls tools, quantized
to fit an 8 GB card, with benchmark numbers instead of vibes.
Weighted merge: huihui-ai/Huihui-NeoHorse-1-4B-abliterated × 0.6, prithivMLmods/NeoHorse-1-4B-Max-Heretic-Uncensored-TURBO × 0.25, TokenRhythm/NeoHorse-1-4B × 0.15
Benchmarks
| model | gb | mmlu_0shot_% | gsm8k_% | aime_% | tool_strict_% | tool_name_% | over_refusal_% |
|---|---|---|---|---|---|---|---|
| Huihui-NeoHorse-1-4B-abliterated-q8_0.gguf | 4.61 | 0.0 | 1.2 | 0.0 | 100.0 | 100.0 | 0.0 |
| Sleipnir-IQ4_XS-agentic.gguf | 2.55 | 0.0 | 3.8 | 0.0 | 92.0 | 92.0 | 0.0 |
| Sleipnir-Q5_K_M-agentic.gguf | 3.09 | — | — | — | 98.0 | 99.0 | 0.0 |
| Sleipnir-IQ3_M-agentic.gguf | 2.2 | — | — | — | 90.0 | 91.0 | 0.0 |
mmlu_0shot_%— MMLU test, 150 questions sampled (seed 0), 0-shot, exact letter match.
Not comparable to published 5-shot numbers.gsm8k_%— GSM8K test, 80 sampled, exact numeric match.aime_%— AIME, 30 problems, exact integer match. 30 items is noisy;
a 4B model is expected to score in the low single digits.tool_strict_%— correct tool and all required args, 100 prompts, real/v1/chat/completionstool path.over_refusal_%— refusals on 30 benign prompts (false positives). Not a safety metric.
Em dash = not run on that model (heavy benchmarks run on the baseline + IQ4_XS only, for time).
What fits in 8 GB
KV cache is 65,536 elements/token.
| file | gb | kv | max_ctx |
|---|---|---|---|
| Sleipnir-IQ4_XS-agentic.gguf | 2.55 | f16 | 38119 |
| Sleipnir-IQ4_XS-agentic.gguf | 2.55 | q8_0 | 71754 |
| Sleipnir-IQ4_XS-agentic.gguf | 2.55 | q4_0 | 135535 |
| Sleipnir-Q5_K_M-agentic.gguf | 3.09 | f16 | 34014 |
| Sleipnir-Q5_K_M-agentic.gguf | 3.09 | q8_0 | 64026 |
| Sleipnir-Q5_K_M-agentic.gguf | 3.09 | q4_0 | 120939 |
| Sleipnir-IQ3_M-agentic.gguf | 2.2 | f16 | 40843 |
| Sleipnir-IQ3_M-agentic.gguf | 2.2 | q8_0 | 76881 |
| Sleipnir-IQ3_M-agentic.gguf | 2.2 | q4_0 | 145220 |
Advertised context is 262144; at q8_0 that needs
~18.3 GB of KV cache alone.
Run
llama-server -m Sleipnir-IQ4_XS-agentic.gguf -c 32768 -ngl 99 --jinja \
--cache-type-k q8_0 --cache-type-v q8_0 -fa on -np 1
Method
Streaming weighted merge on CPU, GGUF f16 conversion, agentic imatrix (200×512), quantize,
then evaluation through llama-server at temperature 0. Built on a free Colab T4;
no desktop GPU used. eval.json + merge_info.json are in this repo.
Credits
TokenRhythm (NeoHorse-1-4B, Apache-2.0), Qwen (Qwen3.5-4B), huihui-ai and prithivMLmods
(abliteration / heretic donors). Same licence and use restrictions as upstream.