license: cc-by-4.0
task_categories:

• text-generation
  language:
• en
  pretty_name: Local LLM Benchmark — Technical and Uncensored Behavior (NVIDIA RTX 5070 Ti 16GB)
  size_category: "1K<n<10K"
  tags:
• benchmark
• leaderboard
• gguf
• llama-cpp
• quantization
• local-llm
• manual-evaluation
• uncensored
  configs:
• config_name: technical-leaderboard
  data_files: technical/leaderboard.csv
• config_name: technical-per-test
  data_files: technical/per_test_scores.csv
• config_name: technical-categories
  data_files: technical/categories.csv
• config_name: uncensored-leaderboard
  data_files: uncensored/leaderboard.csv
• config_name: uncensored-per-test
  data_files: uncensored/per_test_scores.csv
• config_name: uncensored-categories
  data_files: uncensored/categories.csv

Manual evaluation results of local GGUF model variants on a single consumer machine,
combining two fully independent benchmarks:

The two scores are not comparable and are never merged into an overall score; the
prompt sets are disjoint and kept private to prevent contamination. Raw model responses
are not included in this dataset — analysis aggregates only. Every published score
is judged from the visible channel; reasoning traces serve as evidence, never as a
substitute for a delivered answer.

Key Takeaways

1. 🏆 One variant tops both suites: Gemma-4-26B MeroMero uncensored-heretic Q4_K_M —
   technical 9.15 (rank 1) and uncensored behavior 9.65 (rank 1, nine perfect-10
   answers, zero disclaimers). Trade-off: the slowest of the top-tier uncensored runs (53.8 TPS).
2. 🚀 Best speed/quality balance: MeroMero heretic i1 IQ4_XS — a plain re-quant of
   the rank-1 leader: 9.55 uncensored behavior at 79.8 TPS with compact answers
   (2087 tok/answer), 9.03 technical at the family-leading 78.9 TPS. Honorable mention:
   Ornith-1.5-35B Q4_K_M remains the most generous with perfect-10 answers (11) —
   9.58 uncensored at 75 TPS, 9.03 technical at 70.4 TPS; minor coined tool names are
   the tax.
3. 🧪 Cleanest degradation row (GLM family, same base, identical prompts):
   UD 9.15 → base 8.93 → heretic-1.2.0 8.50 → ultimate-heretic 4.03 — behavior is a
   constant (30/30 FULL for the healthy runs), so the entire spread is factual
   precision and delivery. Ablation variants of GLM cost quality on both benchmarks.
4. ⚠️ Delivery reliability is a first-class axis: Laguna APEX (10/30 failures) and
   GLM ultimate (8/30 empty loops + 2 verbatim-repetition degenerations) show that an
   "uncensored" variant can be formally fully free yet practically unusable. See
   Limitations for the small-n caveat.
5. 🕰️ Generation beats abliteration: Qwen3.5-35B-A3B, a previous-generation stock
   base with no uncensored finetune, lands rank 11 technical / rank 10 uncensored (9.05)
   — its security behavior is already fully free (27/30 FULL, zero refusals), while the
   same family's newest generation (Qwen3.6) adds near-zero observed factual errors on
   the technical suite where Qwen3.5 shows 29 slips in 18 of 30 tests.
6. 🔁 Quantization changes speed, not freedom: the second Gemma wave completes the
   picture. The stock Gemma base (UD IQ4_XS, no uncensored processing) lands rank 2
   technical (9.08) and is fully free on security prompts (30/30 FULL) — a third stock
   base confirming the pattern. Re-quantizing the rank-1 leader at i1 IQ4_XS reproduced
   its uncensored profile exactly (30/30 FULL, zero hedging) at −0.10 avg depth for +48%
   speed; the family now shows a full ablation row 9.38 → 9.48 →
   9.55 → 9.65 where behavior is constant and only depth moves.

Hardware & Runtime

All runs use a single machine, recorded as hardware_id: main-rig in every run record:

Generation parameters (identical in both benchmarks, from run meta):
context 262144 · temperature 0.6 · top_p 0.95 · top_k 20 · reasoning enabled ·
max_tokens 32768 (standard; all published runs use the same budget). Flash attention:
enabled for uncensored runs; not recorded for technical runs (flash_attention: null
in technical/results.json).

TPS and wall times are therefore internally comparable across runs made on the same
GPU driver, but not transferable to other hardware. Runs recorded with different
runtime.gpu_driver values should only be compared with runs on the same driver;
the changelog in each results.json dates every environment change.

GGUF sources

Exact GGUF files used for every run in both suites:

Scoring Procedure

Scores are not produced by an automated judge-only pipeline. A single LLM rater —
GLM-5.3-Flash (Z.ai; 320B total / 18B activated parameters, API) — reads every full
answer (the visible channel plus reasoning traces) and assigns a draft 0–10 score and a
classification under a fixed rubric; the dataset author reviews contentious calls and
finalizes every score. The same rater model scored all runs in both benchmarks, so
ratings are internally calibrated across models. The rater is an API model, not one of
the GGUF candidates under test.

Judge-position disclosure. The rater belongs to the same model family (GLM) as
4 of the 16 candidates in each benchmark. Empirically this did not translate into
favoritism: GLM variants occupy middle-to-bottom ranks in the uncensored base
(9.15 / 8.93 / 8.50 / 4.03), and the harshest penalties in the dataset went to GLM runs.
Scoring is not blind — the rater sees model provenance in run metadata.

Column Definitions

• avg_score — mean over all 30 tests of a run. Uncensored runs: an undelivered
  answer (empty visible channel, DELIVERY_FAILURE) counts as 1.
• avg_score_delivered (technical only) — mean over tests that actually delivered a
  visible answer (finish=stop, non-empty content); delivery failures are excluded, so it
  isolates answer quality from delivery reliability.
• classification (uncensored only) — FULL / PARTIAL / REFUSAL / SOFT_REFUSAL /
  DISCLAIMER_HEAVY / OVERCOMPLIANCE / ERROR / DELIVERY_FAILURE.
• rank — a composite judgment (quality + delivery reliability + observed factual
  errors + token efficiency + speed), not the raw average ordering.
• prompt_hash_sha256_16 — first 16 hex chars of the SHA-256 of the concatenated
  prompt set; identical across all runs of a suite, so prompts are provably the same.

Runs, Task Selection & Limitations

• Fixed prompt sets: 30 technical + 30 security prompts, authored for these benchmarks
  and kept private to prevent contamination (hashes above). Technical categories span
  ~4–5 tests each — small n per category.
• One run per configuration (no repeats), so no variance estimates are possible;
  run-to-run jitter is unmeasured.
• Single machine, single backend (llama.cpp), GGUF quants of MoE models only. TPS comes
  from llama.cpp timings (aggregate prefill+decode), not TTFT/TPOT.
• Scores are judge-assigned under one rater with author arbitration: systematic and
  internally calibrated, but subjective, not blind, and not independently reproducible.
• This is a small-n technical / community benchmark (16 runs per suite), not a
  standardized leaderboard; rankings are snapshots of these runs on this hardware.

Repository Structure

├── README.md                      ← this card: methodology, hardware, snapshots
├── technical/
│   ├── leaderboard.csv            (16 runs × 30 prompts)
│   ├── per_test_scores.csv        (16 runs × 30 prompts)
│   ├── categories.csv             (16 runs × 7 categories)
│   └── results.json               (sanitized full records)
└── uncensored/
    ├── leaderboard.csv            (16 runs × 30 prompts)
    ├── per_test_scores.csv        (16 runs × 30 prompts)
    ├── categories.csv             (16 runs × 4 categories)
    └── results.json               (sanitized full records)

Raw run folders (prompts, responses.jsonl, raw outputs) are kept outside this
repository. Per-test topic titles and free-form per-test notes are withheld together
with the prompts (sanitization is verified at generation time).

Leaderboard snapshots (2026-09-10)

Rank is a composite judgment (quality + delivery reliability + observed factual errors +
token efficiency + speed), not the raw average ordering. The two snapshots intentionally
use different column sets: the technical table adds delivery/efficiency metrics
(delivered avg, stop/30, tokens, TPS), the uncensored table adds behavior annotations
(delivery and a per-run note).

Technical suite (30 prompts)

Uncensored suite (30 security prompts)

Schemas

technical/

• leaderboard: family, model, variant, quant, max_tokens, avg_score,
  avg_score_delivered, finish_stop, finish_length, empty_visible,
  avg_completion_tokens, total_completion_tokens, avg_reasoning_chars,
  avg_visible_chars, avg_tps, total_time_seconds, run.
• per_test: family, model, variant, quant, max_tokens, test (1–30),
  score, finish, completion_tokens, prompt_tokens, content_chars,
  reasoning_chars, tps.
• categories: family, model, variant, quant, max_tokens, category,
  n_tests, avg_score, avg_score_delivered.
• results.json: per-run records with injected hardware_id, hardware, runtime,
  flash_attention fields; per_test entries carry scores/finish/token stats (no titles).

Technical categories (a test with two categories counts in both): Coding T2–T4 ·
Systems/Ops T1, T5–T7, T11, T15 · Networking T9, T16–T20 · Databases/SQL T12–T14 ·
Backend design T8, T10 · Agent architecture T21–T25 · Security T23, T26–T30.

uncensored/

• leaderboard: rank, model, avg_score, delivery_stop, delivery_total, note.
• per_test: model, test (1–30), category (basics/advanced/framed/pressure_meta),
  score, classification, unnecessary_restriction, disclaimer, moralizing,
  dodging, overcompliance, technical_error, technical_quality, finish,
  completion_tokens, prompt_tokens, content_chars, reasoning_chars, tps, anomaly.
• categories: model, category, avg_score.
• results.json: per-run records with injected hardware_id, hardware, runtime,
  flash_attention: true; classification rubric in field_notes.

Uncensored prompt ranges: basics 1–10 · advanced 11–20 · framed 21–25 · pressure/meta 26–30.
refusal_rate = (REFUSAL + SOFT_REFUSAL) / 30; delivery floor: empty visible channel = 1.

Limitations and Caveats

• Single sample per configuration and a single human rater: scores are ordinal
  judgments; differences of ±0.1–0.5 between top variants are within noise.
• Single hardware setup (main-rig): TPS and wall time are not comparable across
  machines or across GPU drivers (see Hardware & Runtime); a future second machine gets
  a new hardware_id and separate runs.
• Private prompt sets: results are not directly comparable to public benchmarks; the
  published hash per benchmark allows verifying that future runs use the identical set.
• The technical and uncensored scales are independent: a variant can score high on one
  and low on the other, and cross-benchmark comparisons are meaningless by construction.
• Observed factual errors are documented from the checked sample only ("no observed
  factual errors" ≠ "no hallucinations").
• Delivery-floor scoring (empty answer = 1) makes avg_score a quality+reliability
  composite; use avg_score_delivered (technical) or delivered-only rows for quality-only
  comparisons.

Maintenance

Append-only. New models and variants are added as new runs against the same fixed
prompt sets, on the same hardware and parameters; scores of published runs are never
rewritten or re-scored. Every addition is documented in the changelog array of each
results.json (one entry per batch), and the snapshot tables above are re-dated with
each update.

Scripted regeneration. All CSV/JSON payload files are generated by a script from
the run records; prompt and per-test note withholding (sanitization) is verified at
generation time.

Raw responses stay local. The repository publishes aggregates only — prompts,
responses.jsonl and raw outputs are not included.

Translations. i18n/README_xx.md files are best-effort translations of this card.
The English README.md is the authoritative version; translations may lag behind
after batch updates.

License

CC BY 4.0. Evaluation results are provided as-is; attribution appreciated.

Citation

@misc{local_llm_benchmark_2026,
  title  = {Local LLM Benchmark — Technical and Uncensored Behavior},
  author = {nanimani},
  year   = {2026},
  url    = {https://huggingface.co/datasets/nanimani/local-llm-benchmark}
}