Get the app
misc 0 0 0

shb777/Llama-3.3-8B-Instruct-128K-Evals

published by @shb777

Alignment dataset tracked in the /datasets sub-catalog.

Lifetime downloads
197
Last 30 days
20
Likes
3
Size
256 KB
Created on HF
2025-12-31
Age
9 months ago

Description

[!WARNING]

Always take evals with a big grain of salt. Check the correct metric in each eval in detail for the full picture and do not believe charts blindly.

Evaluation Details

Framework: lm-evaluation-harness

Eval Task: Leaderboard

Models: shb777/Llama-3.3-8B-Instruct-128K and meta-llama/Llama-3.1-8B-Instruct

Hardware: 4 vCPU, 16 GB RAM, H200 GPU

Total Eval Time: 3h 2m

Results

[!WARNING]

Different evals use different metrics, IFEval uses avg(prompt_strict +… See the full description on the dataset page: https://huggingface.co/datasets/shb777/Llama-3.3-8B-Instruct-128K-Evals.

Tags

license:llama3.3region:us

README current version from Hugging Face

license: llama3.3
base_model:

  • shb777/Llama-3.3-8B-Instruct-128K

[!WARNING]
Always take evals with a big grain of salt. Check the correct metric in each eval in detail for the full picture and do not believe charts blindly.

Evaluation Details

Results

[!WARNING]
Different evals use different metrics, IFEval uses avg(prompt_strict + inst_strict), MATH-Hard uses exact_match, BBH, GPQA, MuSR use acc_norm and MMLU-Pro uses acc.

  • Llama 3.3 8B: 45.4% ±2.2% [43.1%, 47.6%]
  • Llama 3.1 8B: 42.3% ±2.3% [40.0%, 44.6%]
Task Llama 3.3 8B Llama 3.1 8B
BBH 54.1% ±1.2% 50.6% ±1.2%
GPQA 29.9% ±2.6% 29.3% ±2.6%
MMLU Pro 38.0% ±0.9% 37.7% ±0.9%
MuSR 37.8% ±3.4% 38.8% ±3.4%
IFEval 85.2% ±3.2% 77.6% ±3.7%
MATH Hard 27.3% ±2.2% 19.8% ±2.0%
Click to expand detailed task results

BBH (Big Bench Hard) Tasks

Task Llama 3.3 8B Llama 3.1 8B
Bbh Boolean Expressions 84.0% ±4.6% 82.0% ±4.8%
Bbh Causal Judgement 58.3% ±7.1% 56.7% ±7.1%
Bbh Date Understanding 52.8% ±6.2% 49.6% ±6.2%
Bbh Disambiguation Qa 55.2% ±6.2% 56.8% ±6.2%
Bbh Formal Fallacies 56.4% ±6.2% 54.8% ±6.2%
Bbh Geometric Shapes 44.0% ±6.2% 28.4% ±5.6%
Bbh Hyperbaton 65.2% ±5.9% 65.2% ±5.9%
Bbh Logical Deduction Five Objects 47.2% ±6.2% 38.4% ±6.0%
Bbh Logical Deduction Seven Objects 45.2% ±6.2% 38.8% ±6.1%
Bbh Logical Deduction Three Objects 71.2% ±5.6% 62.8% ±6.0%
Bbh Movie Recommendation 59.6% ±6.1% 50.8% ±6.2%
Bbh Navigate 65.2% ±5.9% 59.2% ±6.1%
Bbh Object Counting 36.0% ±6.0% 32.8% ±5.8%
Bbh Penguins In A Table 49.3% ±8.1% 45.2% ±8.1%
Bbh Reasoning About Colored Objects 64.0% ±6.0% 66.0% ±5.9%
Bbh Ruin Names 78.4% ±5.1% 63.6% ±6.0%
Bbh Salient Translation Error Detection 53.2% ±6.2% 52.4% ±6.2%
Bbh Snarks 63.5% ±7.1% 62.4% ±7.1%
Bbh Sports Understanding 74.8% ±5.4% 74.4% ±5.4%
Bbh Temporal Sequences 51.6% ±6.2% 44.8% ±6.2%
Bbh Tracking Shuffled Objects Five Objects 23.2% ±5.2% 24.0% ±5.3%
Bbh Tracking Shuffled Objects Seven Objects 21.2% ±5.1% 26.0% ±5.4%
Bbh Tracking Shuffled Objects Three Objects 25.2% ±5.4% 36.0% ±6.0%
Bbh Web Of Lies 54.4% ±6.2% 46.4% ±6.2%

GPQA (Graduate-Level Google-Proof Q&A) Tasks

Task Llama 3.3 8B Llama 3.1 8B
Gpqa Diamond 29.3% ±6.4% 30.8% ±6.4%
Gpqa Extended 30.4% ±3.9% 29.9% ±3.8%
Gpqa Main 29.7% ±4.2% 27.9% ±4.2%

MATH Hard Tasks

Task Llama 3.3 8B Llama 3.1 8B
Math Algebra Hard 53.7% ±5.6% 39.7% ±5.5%
Math Counting And Prob Hard 20.3% ±7.1% 11.4% ±5.6%
Math Geometry Hard 13.6% ±5.9% 10.6% ±5.3%
Math Intermediate Algebra Hard 7.1% ±3.0% 5.7% ±2.7%
Math Num Theory Hard 25.3% ±6.9% 12.3% ±5.2%
Math Prealgebra Hard 43.5% ±7.0% 35.2% ±6.8%
Math Precalculus Hard 7.4% ±4.4% 6.7% ±4.2%

MuSR (Multistep Soft Reasoning) Tasks

Task Llama 3.3 8B Llama 3.1 8B
Musr Murder Mysteries 51.6% ±6.2% 56.0% ±6.2%
Musr Object Placements 25.0% ±5.3% 30.1% ±5.6%
Musr Team Allocation 37.2% ±6.0% 30.4% ±5.7%
Click to expand tasks with significant improvements

MATH Hard Subtasks:

  • Math Algebra Hard: 39.7% → 53.7% (+14.0%)
  • Math Num Theory Hard: 12.3% → 25.3% (+13.0%)
  • Math Counting And Prob Hard: 11.4% → 20.3% (+8.9%)
  • Math Prealgebra Hard: 35.2% → 43.5% (+8.3%)
  • Math Geometry Hard: 10.6% → 13.6% (+3.0%)

BBH Subtasks:

  • Bbh Geometric Shapes: 28.4% → 44.0% (+15.6%)
  • Bbh Ruin Names: 63.6% → 78.4% (+14.8%)
  • Bbh Logical Deduction Five Objects: 38.4% → 47.2% (+8.8%)
  • Bbh Movie Recommendation: 50.8% → 59.6% (+8.8%)
  • Bbh Logical Deduction Three Objects: 62.8% → 71.2% (+8.4%)
  • Bbh Web Of Lies: 46.4% → 54.4% (+8.0%)
  • Bbh Temporal Sequences: 44.8% → 51.6% (+6.8%)
  • Bbh Logical Deduction Seven Objects: 38.8% → 45.2% (+6.4%)
  • Bbh Navigate: 59.2% → 65.2% (+6.0%)
  • Bbh Penguins In A Table: 45.2% → 49.3% (+4.1%)
  • Bbh Date Understanding: 49.6% → 52.8% (+3.2%)
  • Bbh Object Counting: 32.8% → 36.0% (+3.2%)
  • Bbh Boolean Expressions: 82.0% → 84.0% (+2.0%)
Click to expand tasks where Llama 3.1 leads
  • Bbh Tracking Shuffled Objects Three Objects: 36.0% vs 25.2% (Llama 3.1 leads by 10.8%)
  • Musr Object Placements: 30.1% vs 25.0% (Llama 3.1 leads by 5.1%)
  • Bbh Tracking Shuffled Objects Seven Objects: 26.0% vs 21.2% (Llama 3.1 leads by 4.8%)
  • Musr Murder Mysteries: 56.0% vs 51.6% (Llama 3.1 leads by 4.4%)
  • Bbh Reasoning About Colored Objects: 66.0% vs 64.0% (Llama 3.1 leads by 2.0%)
  • Bbh Disambiguation Qa: 56.8% vs 55.2% (Llama 3.1 leads by 1.6%)
  • Gpqa Diamond: 30.8% vs 29.3% (Llama 3.1 leads by 1.5%)
  • Bbh Tracking Shuffled Objects Five Objects: 24.0% vs 23.2% (Llama 3.1 leads by 0.8%)
Aggregate Comparison

README history 1 revision

Every night we snapshot the README of every dataset in the catalog. When the SHA changes we archive the new version and diff it against the last. This is the evolving thought record of the alignment-data field: what the author decided to say about the corpus, and how that framing shifted over time.

  1. 2025-12-31Super-squash branch 'main' using huggingface_huba603ab56 KB
    Loading...

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Related datasets same stage · misc

nvidia/Aegis-AI-Content-Safety-Dataset-2.0
99.3K dl 114 likes 27.1 MB
🛡️ Nemotron Content Safety Dataset V2 The Nemotron Content Safety Dataset V2, formerly known as Aegis AI Content Safety Data...
allenai/tulu-3-sft-personas-instruction-following
88.4K dl 70 likes 37.4 MB
Dataset Descriptions This dataset contains 29980 examples and is synthetically created to enhance model's capabilities to follo...
meta-llama/Llama-3.1-8B-Instruct-evals
73.2K dl 36 likes 232.9 MB
Dataset Card for Llama-3.1-8B-Instruct Evaluation Result Details This dataset contains the Meta evaluation result details for...
Manusagents/GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-5-Mythos-5-Qwen-3.7-Max-and-more-Distillation-Dataset
58.3K dl 226 likes 10.8 GB
📖 The Open Distillation Codex 🌌 The Ultimate Open-Source Distillation Dataset — No Skip, Full, with Attack & Defe...
nvidia/Aegis-AI-Content-Safety-Dataset-1.0
36.1K dl 63 likes 5.2 MB
🛡️ Nemotron Content Safety Dataset V1 Nemotron Content Safety Dataset V1, formerly known as Aegis AI Content Safety Dataset, i...
innodatalabs/rt-gsm8k-gaia
35.9K dl 15 KB
rt-gsm8k dataset Red teaming gsm8k-gaia dataset. Generated from the "GSM8K" dataset and "GAIA" dataset. Dataset Str...
HuggingFaceH4/hhh_alignment
35.3K dl 27 likes 266 KB
This task evaluates language models on alignment, broken down into categories of helpfulness, honesty/accuracy, harmlessness, and ...
ethz-spylab/rlhf_trojan_dataset
26.2K dl 6 likes 29.9 MB
Dataset for the competition This is the official dataset for the competition "Find the Trojan: Universal Backdoor Detection in ...

Read further

Community best practices (architecture)
4 rules for MoE, hybrid SSM, FP8 requantization, and Gemma quirks.
The six workflow stages
What misc is doing more work than its name suggests.
Beyond English: multilingual corpora
The four language communities and their workarounds.
Non-English abliteration communities
Deep dive: huihui-ai, layer-window, slop-framing, CJK tokenizer fixes.

Metadata

License
llama3.3

Repository

FILE .gitattributes FILE README.md FILE results_llama_3_1_8b_instruct_2025-12-31T10-42-16.731213.json FILE results_llama_3_3_8b_instruct_2025-12-31T09-07-28.409075.json

File contents live on Hugging Face. Click a file or folder to open it there.

Download the dataset

0

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python) 
from datasets import load_dataset
ds = load_dataset("shb777/Llama-3.3-8B-Instruct-128K-Evals")
Raw snapshot (Python) 
from huggingface_hub import snapshot_download
snapshot_download(repo_id="shb777/Llama-3.3-8B-Instruct-128K-Evals", repo_type="dataset")
git clone (requires git-lfs) 
git clone https://huggingface.co/datasets/shb777/Llama-3.3-8B-Instruct-128K-Evals
Browse repository on Hugging Face ->
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Get the app Browse the catalog
Open in Abliteration