Get the app
evaluate 0 0 0

tytodd/qwen3.5-2b-v2-smoke

published by @tytodd

Alignment dataset tracked in the /datasets sub-catalog.

Lifetime downloads
797
Last 30 days
122
Likes
-
Size
33.7 MB
Created on HF
2026-04-19
Age
6 months ago

Description

qwen3.5-2b-v2-smoke

Repo: tytodd/qwen3.5-2b-v2-smoke

Model: Qwen/Qwen3.5-2B

Config: /tmp/v2-smoke.yaml

benchmark

train

val

ood

all

chatbot_arena_conversations

70.00%

60.00%

65.00%

hh_rlhf

50.00%

50.00%

50.00%

ultrafeedback

60.00%

60.00%

60.00%

projudgebench

90.00%

90.00%

90.00%

reward_bench_2

70.00%

80.00%

75.00%

aes2_essay_scoring

40.00%

30.00%

35.00%

halueval_qa

80.00%

90.00%

85.00%

halueval_dialogue

40.00%

70.00%

55.00%

or_bench_80k

20.00%… See the full description on the dataset page: https://huggingface.co/datasets/tytodd/qwen3.5-2b-v2-smoke.

Tags

size_categories:n<1Kformat:parquetformat:optimized-parquetmodality:textlibrary:datasetslibrary:pandaslibrary:polarslibrary:mlcroissantregion:us

README current version from Hugging Face

dataset_info:

  • config_name: aes2_essay_scoring
    features:
    • name: input
      struct:
      • name: full_text
        dtype: string
    • name: prediction
      struct:
      • name: score
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 976965
      num_examples: 10
    • name: val
      num_bytes: 308721
      num_examples: 10
      download_size: 1330092
      dataset_size: 1285686
  • config_name: anli_r1
    features:
    • name: input
      struct:
      • name: hypothesis
        dtype: string
      • name: premise
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 201314
      num_examples: 10
    • name: val
      num_bytes: 240343
      num_examples: 10
      download_size: 468205
      dataset_size: 441657
  • config_name: anli_r2
    features:
    • name: input
      struct:
      • name: hypothesis
        dtype: string
      • name: premise
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 219386
      num_examples: 10
    • name: val
      num_bytes: 251084
      num_examples: 10
      download_size: 485917
      dataset_size: 470470
  • config_name: anli_r3
    features:
    • name: input
      struct:
      • name: hypothesis
        dtype: string
      • name: premise
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 326455
      num_examples: 10
    • name: val
      num_bytes: 287161
      num_examples: 10
      download_size: 649098
      dataset_size: 613616
  • config_name: arc_challenge
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 50966
      num_examples: 3
      download_size: 72133
      dataset_size: 50966
  • config_name: argument_quality_ranking
    features:
    • name: input
      struct:
      • name: argument
        dtype: string
      • name: topic
        dtype: string
    • name: prediction
      struct:
      • name: quality_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 57227
      num_examples: 3
      download_size: 72756
      dataset_size: 57227
  • config_name: big_patent_innovation
    features:
    • name: input
      struct:
      • name: description
        dtype: string
    • name: prediction
      struct:
      • name: innovation_score
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 314088
      num_examples: 10
    • name: val
      num_bytes: 283292
      num_examples: 10
      download_size: 667359
      dataset_size: 597380
  • config_name: boardgame_qa
    features:
    • name: input
      struct:
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: answer
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 356222
      num_examples: 10
    • name: val
      num_bytes: 381807
      num_examples: 10
      download_size: 782236
      dataset_size: 738029
  • config_name: chatbot_arena_conversations
    features:
    • name: input
      struct:
      • name: question
        dtype: string
      • name: response_A
        dtype: string
      • name: response_B
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 251026
      num_examples: 10
    • name: val
      num_bytes: 246629
      num_examples: 10
      download_size: 540987
      dataset_size: 497655
  • config_name: civil_comments
    features:
    • name: input
      struct:
      • name: comment
        dtype: string
    • name: prediction
      struct:
      • name: toxicity_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 154986
      num_examples: 10
    • name: val
      num_bytes: 201921
      num_examples: 10
      download_size: 381787
      dataset_size: 356907
  • config_name: code_judge_bench
    features:
    • name: input
      struct:
      • name: code_A
        dtype: string
      • name: code_B
        dtype: string
      • name: problem
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 2928002
      num_examples: 3
      download_size: 1663680
      dataset_size: 2928002
  • config_name: colbert_humor_detection
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: humor_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 172391
      num_examples: 10
    • name: val
      num_bytes: 187374
      num_examples: 10
      download_size: 383159
      dataset_size: 359765
  • config_name: customer_support_tickets_en
    features:
    • name: input
      struct:
      • name: body
        dtype: string
      • name: subject
        dtype: string
    • name: prediction
      struct:
      • name: queue
        dtype: string
      • name: type
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 1041566
      num_examples: 10
    • name: val
      num_bytes: 294709
      num_examples: 10
      download_size: 1381506
      dataset_size: 1336275
  • config_name: customer_support_tickets_gorkem
    features:
    • name: input
      struct:
      • name: ticket_text
        dtype: string
    • name: prediction
      struct:
      • name: ticket_subject
        dtype: string
      • name: ticket_type
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 245522
      num_examples: 10
    • name: val
      num_bytes: 266482
      num_examples: 10
      download_size: 549512
      dataset_size: 512004
  • config_name: dbpedia_easy
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: l1_class
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 195283
      num_examples: 10
    • name: val
      num_bytes: 935190
      num_examples: 10
      download_size: 1156102
      dataset_size: 1130473
  • config_name: dbpedia_hard
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: l1_class
        dtype: string
      • name: l2_class
        dtype: string
      • name: l3_class
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 1365969
      num_examples: 10
    • name: val
      num_bytes: 430985
      num_examples: 10
      download_size: 1382087
      dataset_size: 1796954
  • config_name: dbpedia_medium
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: l1_class
        dtype: string
      • name: l2_class
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 333294
      num_examples: 10
    • name: val
      num_bytes: 325586
      num_examples: 10
      download_size: 695625
      dataset_size: 658880
  • config_name: enron_email_quality
    features:
    • name: input
      struct:
      • name: body
        dtype: string
      • name: subject
        dtype: string
    • name: prediction
      struct:
      • name: quality_score
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 517765
      num_examples: 10
    • name: val
      num_bytes: 557950
      num_examples: 10
      download_size: 820138
      dataset_size: 1075715
  • config_name: enron_email_type
    features:
    • name: input
      struct:
      • name: body
        dtype: string
      • name: subject
        dtype: string
    • name: prediction
      struct:
      • name: email_type
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 391297
      num_examples: 10
    • name: val
      num_bytes: 506696
      num_examples: 10
      download_size: 695420
      dataset_size: 897993
  • config_name: enron_reply_quality
    features:
    • name: input
      struct:
      • name: original_email
        dtype: string
      • name: reply
        dtype: string
    • name: prediction
      struct:
      • name: quality
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 576939
      num_examples: 10
    • name: val
      num_bytes: 570383
      num_examples: 10
      download_size: 856891
      dataset_size: 1147322
  • config_name: go_emotions
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: labels
        list: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 256683
      num_examples: 10
    • name: val
      num_bytes: 203047
      num_examples: 10
      download_size: 483411
      dataset_size: 459730
  • config_name: gpqa_diamond
    features:
    • name: input
      struct:
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 164148
      num_examples: 3
      download_size: 175560
      dataset_size: 164148
  • config_name: halueval_dialogue
    features:
    • name: input
      struct:
      • name: dialogue_history
        dtype: string
      • name: knowledge
        dtype: string
      • name: response
        dtype: string
    • name: prediction
      struct:
      • name: hallucination
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 293368
      num_examples: 10
    • name: val
      num_bytes: 1650718
      num_examples: 10
      download_size: 1349383
      dataset_size: 1944086
  • config_name: halueval_qa
    features:
    • name: input
      struct:
      • name: answer
        dtype: string
      • name: knowledge
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: hallucination
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 321957
      num_examples: 10
    • name: val
      num_bytes: 231709
      num_examples: 10
      download_size: 585928
      dataset_size: 553666
  • config_name: halueval_summarization
    features:
    • name: input
      struct:
      • name: document
        dtype: string
      • name: summary
        dtype: string
    • name: prediction
      struct:
      • name: hallucination
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 140890
      num_examples: 3
      download_size: 165478
      dataset_size: 140890
  • config_name: hh_rlhf
    features:
    • name: input
      struct:
      • name: question
        dtype: string
      • name: response_A
        dtype: string
      • name: response_B
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 286080
      num_examples: 10
    • name: val
      num_bytes: 273013
      num_examples: 10
      download_size: 599023
      dataset_size: 559093
  • config_name: judge_bench
    features:
    • name: input
      struct:
      • name: question
        dtype: string
      • name: response_A
        dtype: string
      • name: response_B
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 170555
      num_examples: 3
      download_size: 214071
      dataset_size: 170555
  • config_name: lex_glue_case_hold
    features:
    • name: input
      struct:
      • name: context
        dtype: string
      • name: option_a
        dtype: string
      • name: option_b
        dtype: string
      • name: option_c
        dtype: string
      • name: option_d
        dtype: string
      • name: option_e
        dtype: string
    • name: prediction
      struct:
      • name: selected_option
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 355619
      num_examples: 10
    • name: val
      num_bytes: 354919
      num_examples: 10
      download_size: 761590
      dataset_size: 710538
  • config_name: lex_glue_ledgar
    features:
    • name: input
      struct:
      • name: provision_text
        dtype: string
    • name: prediction
      struct:
      • name: provision_type
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 261204
      num_examples: 10
    • name: val
      num_bytes: 267914
      num_examples: 10
      download_size: 574475
      dataset_size: 529118
  • config_name: medical_abstracts
    features:
    • name: input
      struct:
      • name: medical_abstract
        dtype: string
    • name: prediction
      struct:
      • name: condition_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 223611
      num_examples: 10
    • name: val
      num_bytes: 298138
      num_examples: 10
      download_size: 568090
      dataset_size: 521749
  • config_name: mfrc
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: annotation
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 256623
      num_examples: 10
    • name: val
      num_bytes: 238432
      num_examples: 10
      download_size: 527864
      dataset_size: 495055
  • config_name: mmlu
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 1432815
      num_examples: 10
    • name: val
      num_bytes: 321016
      num_examples: 10
      download_size: 1785539
      dataset_size: 1753831
  • config_name: mmlu_pro
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 129946
      num_examples: 3
      download_size: 149157
      dataset_size: 129946
  • config_name: mt_bench_human_judgments
    features:
    • name: input
      struct:
      • name: question
        dtype: string
      • name: response_A
        dtype: string
      • name: response_B
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 91591
      num_examples: 3
      download_size: 121846
      dataset_size: 91591
  • config_name: musr_murder_mysteries
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 334825
      num_examples: 3
      download_size: 352281
      dataset_size: 334825
  • config_name: musr_object_placements
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 121268
      num_examples: 3
      download_size: 137984
      dataset_size: 121268
  • config_name: musr_team_allocation
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 174748
      num_examples: 3
      download_size: 198571
      dataset_size: 174748
  • config_name: or_bench_80k
    features:
    • name: input
      struct:
      • name: prompt
        dtype: string
    • name: prediction
      struct:
      • name: or_bench_category
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 240780
      num_examples: 10
    • name: val
      num_bytes: 544015
      num_examples: 10
      download_size: 814013
      dataset_size: 784795
  • config_name: or_bench_hard_1k
    features:
    • name: input
      struct:
      • name: prompt
        dtype: string
    • name: prediction
      struct:
      • name: or_bench_category
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 204226
      num_examples: 10
    • name: val
      num_bytes: 242017
      num_examples: 10
      download_size: 473241
      dataset_size: 446243
  • config_name: or_bench_toxic
    features:
    • name: input
      struct:
      • name: prompt
        dtype: string
    • name: prediction
      struct:
      • name: or_bench_category
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 57236
      num_examples: 3
      download_size: 73964
      dataset_size: 57236
  • config_name: projudgebench
    features:
    • name: input
      struct:
      • name: correct_answer
        dtype: string
      • name: question
        dtype: string
      • name: step_to_evaluate
        dtype: string
      • name: steps
        list: string
    • name: prediction
      struct:
      • name: correct
        dtype: bool
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 502024
      num_examples: 10
    • name: val
      num_bytes: 418312
      num_examples: 10
      download_size: 967798
      dataset_size: 920336
  • config_name: reward_bench_2
    features:
    • name: input
      struct:
      • name: prompt
        dtype: string
      • name: response_A
        dtype: string
      • name: response_B
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 2236389
      num_examples: 10
    • name: val
      num_bytes: 345200
      num_examples: 10
      download_size: 1732554
      dataset_size: 2581589
  • config_name: rod101_essay_scoring
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: score
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: ood
      num_bytes: 114803
      num_examples: 3
      download_size: 140391
      dataset_size: 114803
  • config_name: sem_eval_2010_task_8
    features:
    • name: input
      struct:
      • name: sentence
        dtype: string
    • name: prediction
      struct:
      • name: relation_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 342366
      num_examples: 10
    • name: val
      num_bytes: 291393
      num_examples: 10
      download_size: 666969
      dataset_size: 633759
  • config_name: smollm_corpus
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: audience
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 347469
      num_examples: 10
    • name: val
      num_bytes: 319074
      num_examples: 10
      download_size: 712001
      dataset_size: 666543
  • config_name: snli
    features:
    • name: input
      struct:
      • name: hypothesis
        dtype: string
      • name: premise
        dtype: string
    • name: prediction
      struct:
      • name: label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 209013
      num_examples: 10
    • name: val
      num_bytes: 193065
      num_examples: 10
      download_size: 428935
      dataset_size: 402078
  • config_name: spartqa_mchoice
    features:
    • name: input
      struct:
      • name: choices
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: choice
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 453302
      num_examples: 10
    • name: val
      num_bytes: 515525
      num_examples: 10
      download_size: 1007307
      dataset_size: 968827
  • config_name: toxigen_data
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: toxicity_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 160294
      num_examples: 10
    • name: val
      num_bytes: 164128
      num_examples: 10
      download_size: 347979
      dataset_size: 324422
  • config_name: tweet_eval_emotion
    features:
    • name: input
      struct:
      • name: tweet
        dtype: string
    • name: prediction
      struct:
      • name: emotion_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 159451
      num_examples: 10
    • name: val
      num_bytes: 177255
      num_examples: 10
      download_size: 363215
      dataset_size: 336706
  • config_name: tweet_eval_hate
    features:
    • name: input
      struct:
      • name: tweet
        dtype: string
    • name: prediction
      struct:
      • name: hate_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 208922
      num_examples: 10
    • name: val
      num_bytes: 184479
      num_examples: 10
      download_size: 414229
      dataset_size: 393401
  • config_name: tweet_eval_irony
    features:
    • name: input
      struct:
      • name: tweet
        dtype: string
    • name: prediction
      struct:
      • name: irony_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 203501
      num_examples: 10
    • name: val
      num_bytes: 190622
      num_examples: 10
      download_size: 422553
      dataset_size: 394123
  • config_name: tweet_eval_offensive
    features:
    • name: input
      struct:
      • name: tweet
        dtype: string
    • name: prediction
      struct:
      • name: offensive_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 178766
      num_examples: 10
    • name: val
      num_bytes: 175761
      num_examples: 10
      download_size: 376178
      dataset_size: 354527
  • config_name: tweet_eval_sentiment
    features:
    • name: input
      struct:
      • name: tweet
        dtype: string
    • name: prediction
      struct:
      • name: sentiment_label
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 186798
      num_examples: 10
    • name: val
      num_bytes: 203378
      num_examples: 10
      download_size: 415523
      dataset_size: 390176
  • config_name: ultrafeedback
    features:
    • name: input
      struct:
      • name: prompt
        dtype: string
      • name: response
        dtype: string
    • name: prediction
      struct:
      • name: instruction_following
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 317437
      num_examples: 10
    • name: val
      num_bytes: 244523
      num_examples: 10
      download_size: 615831
      dataset_size: 561960
  • config_name: writingprompts_quality
    features:
    • name: input
      struct:
      • name: prompt
        dtype: string
      • name: story
        dtype: string
    • name: prediction
      struct:
      • name: quality_score
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 273582
      num_examples: 10
    • name: val
      num_bytes: 259726
      num_examples: 10
      download_size: 592972
      dataset_size: 533308
  • config_name: yahoo_answers_quality
    features:
    • name: input
      struct:
      • name: answer
        dtype: string
      • name: question
        dtype: string
    • name: prediction
      struct:
      • name: rating
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 234613
      num_examples: 10
    • name: val
      num_bytes: 236685
      num_examples: 10
      download_size: 498419
      dataset_size: 471298
  • config_name: yelp
    features:
    • name: input
      struct:
      • name: text
        dtype: string
    • name: prediction
      struct:
      • name: rating
        dtype: string
    • name: reasoning
      dtype: string
    • name: messages
      struct:
      • name: messages
        list:
        • name: content
          dtype: string
        • name: role
          dtype: string
      • name: outputs
        struct:
        • name: reasoning_content
          dtype: string
        • name: text
          dtype: string
    • name: correct
      dtype: bool
      splits:
    • name: train
      num_bytes: 208031
      num_examples: 10
    • name: val
      num_bytes: 212241
      num_examples: 10
      download_size: 460340
      dataset_size: 420272
      configs:
  • config_name: aes2_essay_scoring
    data_files:
    • split: train
      path: aes2_essay_scoring/train-*
    • split: val
      path: aes2_essay_scoring/val-*
  • config_name: anli_r1
    data_files:
    • split: train
      path: anli_r1/train-*
    • split: val
      path: anli_r1/val-*
  • config_name: anli_r2
    data_files:
    • split: train
      path: anli_r2/train-*
    • split: val
      path: anli_r2/val-*
  • config_name: anli_r3
    data_files:
    • split: train
      path: anli_r3/train-*
    • split: val
      path: anli_r3/val-*
  • config_name: arc_challenge
    data_files:
    • split: ood
      path: arc_challenge/ood-*
  • config_name: argument_quality_ranking
    data_files:
    • split: ood
      path: argument_quality_ranking/ood-*
  • config_name: big_patent_innovation
    data_files:
    • split: train
      path: big_patent_innovation/train-*
    • split: val
      path: big_patent_innovation/val-*
  • config_name: boardgame_qa
    data_files:
    • split: train
      path: boardgame_qa/train-*
    • split: val
      path: boardgame_qa/val-*
  • config_name: chatbot_arena_conversations
    data_files:
    • split: train
      path: chatbot_arena_conversations/train-*
    • split: val
      path: chatbot_arena_conversations/val-*
  • config_name: civil_comments
    data_files:
    • split: train
      path: civil_comments/train-*
    • split: val
      path: civil_comments/val-*
  • config_name: code_judge_bench
    data_files:
    • split: ood
      path: code_judge_bench/ood-*
  • config_name: colbert_humor_detection
    data_files:
    • split: train
      path: colbert_humor_detection/train-*
    • split: val
      path: colbert_humor_detection/val-*
  • config_name: customer_support_tickets_en
    data_files:
    • split: train
      path: customer_support_tickets_en/train-*
    • split: val
      path: customer_support_tickets_en/val-*
  • config_name: customer_support_tickets_gorkem
    data_files:
    • split: train
      path: customer_support_tickets_gorkem/train-*
    • split: val
      path: customer_support_tickets_gorkem/val-*
  • config_name: dbpedia_easy
    data_files:
    • split: train
      path: dbpedia_easy/train-*
    • split: val
      path: dbpedia_easy/val-*
  • config_name: dbpedia_hard
    data_files:
    • split: train
      path: dbpedia_hard/train-*
    • split: val
      path: dbpedia_hard/val-*
  • config_name: dbpedia_medium
    data_files:
    • split: train
      path: dbpedia_medium/train-*
    • split: val
      path: dbpedia_medium/val-*
  • config_name: enron_email_quality
    data_files:
    • split: train
      path: enron_email_quality/train-*
    • split: val
      path: enron_email_quality/val-*
  • config_name: enron_email_type
    data_files:
    • split: train
      path: enron_email_type/train-*
    • split: val
      path: enron_email_type/val-*
  • config_name: enron_reply_quality
    data_files:
    • split: train
      path: enron_reply_quality/train-*
    • split: val
      path: enron_reply_quality/val-*
  • config_name: go_emotions
    data_files:
    • split: train
      path: go_emotions/train-*
    • split: val
      path: go_emotions/val-*
  • config_name: gpqa_diamond
    data_files:
    • split: ood
      path: gpqa_diamond/ood-*
  • config_name: halueval_dialogue
    data_files:
    • split: train
      path: halueval_dialogue/train-*
    • split: val
      path: halueval_dialogue/val-*
  • config_name: halueval_qa
    data_files:
    • split: train
      path: halueval_qa/train-*
    • split: val
      path: halueval_qa/val-*
  • config_name: halueval_summarization
    data_files:
    • split: ood
      path: halueval_summarization/ood-*
  • config_name: hh_rlhf
    data_files:
    • split: train
      path: hh_rlhf/train-*
    • split: val
      path: hh_rlhf/val-*
  • config_name: judge_bench
    data_files:
    • split: ood
      path: judge_bench/ood-*
  • config_name: lex_glue_case_hold
    data_files:
    • split: train
      path: lex_glue_case_hold/train-*
    • split: val
      path: lex_glue_case_hold/val-*
  • config_name: lex_glue_ledgar
    data_files:
    • split: train
      path: lex_glue_ledgar/train-*
    • split: val
      path: lex_glue_ledgar/val-*
  • config_name: medical_abstracts
    data_files:
    • split: train
      path: medical_abstracts/train-*
    • split: val
      path: medical_abstracts/val-*
  • config_name: mfrc
    data_files:
    • split: train
      path: mfrc/train-*
    • split: val
      path: mfrc/val-*
  • config_name: mmlu
    data_files:
    • split: train
      path: mmlu/train-*
    • split: val
      path: mmlu/val-*
  • config_name: mmlu_pro
    data_files:
    • split: ood
      path: mmlu_pro/ood-*
  • config_name: mt_bench_human_judgments
    data_files:
    • split: ood
      path: mt_bench_human_judgments/ood-*
  • config_name: musr_murder_mysteries
    data_files:
    • split: ood
      path: musr_murder_mysteries/ood-*
  • config_name: musr_object_placements
    data_files:
    • split: ood
      path: musr_object_placements/ood-*
  • config_name: musr_team_allocation
    data_files:
    • split: ood
      path: musr_team_allocation/ood-*
  • config_name: or_bench_80k
    data_files:
    • split: train
      path: or_bench_80k/train-*
    • split: val
      path: or_bench_80k/val-*
  • config_name: or_bench_hard_1k
    data_files:
    • split: train
      path: or_bench_hard_1k/train-*
    • split: val
      path: or_bench_hard_1k/val-*
  • config_name: or_bench_toxic
    data_files:
    • split: ood
      path: or_bench_toxic/ood-*
  • config_name: projudgebench
    data_files:
    • split: train
      path: projudgebench/train-*
    • split: val
      path: projudgebench/val-*
  • config_name: reward_bench_2
    data_files:
    • split: train
      path: reward_bench_2/train-*
    • split: val
      path: reward_bench_2/val-*
  • config_name: rod101_essay_scoring
    data_files:
    • split: ood
      path: rod101_essay_scoring/ood-*
  • config_name: sem_eval_2010_task_8
    data_files:
    • split: train
      path: sem_eval_2010_task_8/train-*
    • split: val
      path: sem_eval_2010_task_8/val-*
  • config_name: smollm_corpus
    data_files:
    • split: train
      path: smollm_corpus/train-*
    • split: val
      path: smollm_corpus/val-*
  • config_name: snli
    data_files:
    • split: train
      path: snli/train-*
    • split: val
      path: snli/val-*
  • config_name: spartqa_mchoice
    data_files:
    • split: train
      path: spartqa_mchoice/train-*
    • split: val
      path: spartqa_mchoice/val-*
  • config_name: toxigen_data
    data_files:
    • split: train
      path: toxigen_data/train-*
    • split: val
      path: toxigen_data/val-*
  • config_name: tweet_eval_emotion
    data_files:
    • split: train
      path: tweet_eval_emotion/train-*
    • split: val
      path: tweet_eval_emotion/val-*
  • config_name: tweet_eval_hate
    data_files:
    • split: train
      path: tweet_eval_hate/train-*
    • split: val
      path: tweet_eval_hate/val-*
  • config_name: tweet_eval_irony
    data_files:
    • split: train
      path: tweet_eval_irony/train-*
    • split: val
      path: tweet_eval_irony/val-*
  • config_name: tweet_eval_offensive
    data_files:
    • split: train
      path: tweet_eval_offensive/train-*
    • split: val
      path: tweet_eval_offensive/val-*
  • config_name: tweet_eval_sentiment
    data_files:
    • split: train
      path: tweet_eval_sentiment/train-*
    • split: val
      path: tweet_eval_sentiment/val-*
  • config_name: ultrafeedback
    data_files:
    • split: train
      path: ultrafeedback/train-*
    • split: val
      path: ultrafeedback/val-*
  • config_name: writingprompts_quality
    data_files:
    • split: train
      path: writingprompts_quality/train-*
    • split: val
      path: writingprompts_quality/val-*
  • config_name: yahoo_answers_quality
    data_files:
    • split: train
      path: yahoo_answers_quality/train-*
    • split: val
      path: yahoo_answers_quality/val-*
  • config_name: yelp
    data_files:
    • split: train
      path: yelp/train-*
    • split: val
      path: yelp/val-*

qwen3.5-2b-v2-smoke

  • Repo: tytodd/qwen3.5-2b-v2-smoke
  • Model: Qwen/Qwen3.5-2B
  • Config: /tmp/v2-smoke.yaml
benchmark train val ood all
chatbot_arena_conversations 70.00% 60.00% 65.00%
hh_rlhf 50.00% 50.00% 50.00%
ultrafeedback 60.00% 60.00% 60.00%
projudgebench 90.00% 90.00% 90.00%
reward_bench_2 70.00% 80.00% 75.00%
aes2_essay_scoring 40.00% 30.00% 35.00%
halueval_qa 80.00% 90.00% 85.00%
halueval_dialogue 40.00% 70.00% 55.00%
or_bench_80k 20.00% 40.00% 30.00%
or_bench_hard_1k 70.00% 40.00% 55.00%
toxigen_data 80.00% 100.00% 90.00%
civil_comments 90.00% 100.00% 95.00%
boardgame_qa 100.00% 100.00% 100.00%
go_emotions 20.00% 30.00% 25.00%
mfrc 0.00% 0.00% 0.00%
tweet_eval_emotion 80.00% 90.00% 85.00%
yelp 60.00% 60.00% 60.00%
tweet_eval_sentiment 70.00% 70.00% 70.00%
mmlu 50.00% 80.00% 65.00%
spartqa_mchoice 80.00% 70.00% 75.00%
anli_r1 90.00% 70.00% 80.00%
anli_r2 90.00% 60.00% 75.00%
anli_r3 40.00% 60.00% 50.00%
snli 70.00% 80.00% 75.00%
sem_eval_2010_task_8 50.00% 30.00% 40.00%
smollm_corpus 30.00% 10.00% 20.00%
medical_abstracts 60.00% 50.00% 55.00%
lex_glue_case_hold 40.00% 70.00% 55.00%
lex_glue_ledgar 60.00% 80.00% 70.00%
dbpedia_easy 100.00% 80.00% 90.00%
dbpedia_medium 60.00% 60.00% 60.00%
dbpedia_hard 30.00% 10.00% 20.00%
colbert_humor_detection 70.00% 60.00% 65.00%
tweet_eval_irony 40.00% 50.00% 45.00%
tweet_eval_hate 80.00% 70.00% 75.00%
tweet_eval_offensive 100.00% 90.00% 95.00%
customer_support_tickets_en 50.00% 30.00% 40.00%
customer_support_tickets_gorkem 0.00% 0.00% 0.00%
yahoo_answers_quality 60.00% 30.00% 45.00%
big_patent_innovation 10.00% 20.00% 15.00%
writingprompts_quality 20.00% 60.00% 40.00%
enron_email_type 50.00% 60.00% 55.00%
enron_email_quality 10.00% 20.00% 15.00%
enron_reply_quality 40.00% 40.00% 40.00%
argument_quality_ranking 33.33% 33.33%
rod101_essay_scoring 0.00% 0.00%
or_bench_toxic 66.67% 66.67%
judge_bench 66.67% 66.67%
musr_team_allocation 33.33% 33.33%
musr_object_placements 33.33% 33.33%
musr_murder_mysteries 33.33% 33.33%
halueval_summarization 100.00% 100.00%
code_judge_bench 33.33% 33.33%
mmlu_pro 100.00% 100.00%
gpqa_diamond 66.67% 66.67%
arc_challenge 100.00% 100.00%
mt_bench_human_judgments 100.00% 100.00%
all 56.14% 56.82% 58.97% 56.58%

README history 20 revisions

Every night we snapshot the README of every dataset in the catalog. When the SHA changes we archive the new version and diff it against the last. This is the evolving thought record of the alignment-data field: what the author decided to say about the corpus, and how that framing shifted over time.

  1. 2026-04-19Update dataset card from inference metrics summary89b49b951.1 KB
    Loading...
  2. 2026-04-19Upload dataset5c65d6748.2 KB
    Loading...
  3. 2026-04-19Upload datasetf0ea48147.3 KB
    Loading...
  4. 2026-04-19Upload dataset739d16d46.6 KB
    Loading...
  5. 2026-04-19Upload datasetebf0c0d45.9 KB
    Loading...
  6. 2026-04-19Upload dataset185df5d45.1 KB
    Loading...
  7. 2026-04-19Upload dataset29cfc0f44.3 KB
    Loading...
  8. 2026-04-19Upload datasetbc0395943.5 KB
    Loading...
  9. 2026-04-19Upload datasetec2164642.8 KB
    Loading...
  10. 2026-04-19Upload dataset8bde04942 KB
    Loading...
  11. 2026-04-19Upload dataset6621d9441.2 KB
    Loading...
  12. 2026-04-19Upload dataset274776d40.4 KB
    Loading...
  13. 2026-04-19Upload dataset2c8819339.7 KB
    Loading...
  14. 2026-04-19Upload dataset732123338.9 KB
    Loading...
  15. 2026-04-19Upload datasetb8d8e2138.2 KB
    Loading...
  16. 2026-04-19Upload dataset876c5f837.3 KB
    Loading...
  17. 2026-04-19Upload dataset70a339536.4 KB
    Loading...
  18. 2026-04-19Upload dataset4c8e8b635.5 KB
    Loading...
  19. 2026-04-19Upload dataset62f87e634.6 KB
    Loading...
  20. 2026-04-19Upload datasetcb0e08c33.7 KB
    Loading...

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Related datasets same stage · evaluate

google/IFEval
1.7M dl 182 likes 210 KB
Dataset Card for IFEval Dataset Summary This dataset contains the prompts used in the Instruction-Following Ev...
JailbreakBench/JBB-Behaviors
418.2K dl 143 likes 992 KB
An Open Robustness Benchmark for Jailbreaking Language Models NeurIPS 2024 Datasets and Benchmarks Track P...
Beijing-AISI/panda-bench
195.6K dl 3.1 GB
PandaBench PandaBench is a comprehensive benchmark for evaluating Large Language Model (LLM) safety, focusing on jailbreak atta...
allenai/wildguardmix
137.3K dl 108 likes 53.4 MB
Dataset Card for WildGuardMix Disclaimer: The data includes examples that might be disturbing, harmful or upsetting....
walledai/HarmBench
135.7K dl 59 likes 109 KB
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal Paper: HarmBench: A Standardized Ev...
walledai/XSTest
108.3K dl 30 likes 22 KB
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paper: XSTest: A Test Suite for Ide...
bench-llm/or-bench
77.2K dl 22 likes 32.1 MB
OR-Bench: An Over-Refusal Benchmark for Large Language Models Please see our demo at HuggingFace Spaces. Overall P...
argilla/magpie-ultra-v1.0
65.5K dl 53 likes 20.7 GB
Dataset Card for magpie-ultra-v1.0 This dataset has been created with distilabel. Dataset Summary magpie...

Read further

Community best practices (evaluation)
4 rules on judges, benchmarks, and reporting.
Reference-set drift over time
Why the same benchmark means different things across model generations.
How we label the method for each model
The 12 classification rules that assign M1-M8 to every model.
Non-English abliteration communities
The Korean scepticism about one-click tools and non-English benchmark blindspots.

Repository

DIR aes2_essay_scoring 2 files DIR anli_r1 2 files DIR anli_r2 2 files DIR anli_r3 2 files DIR arc_challenge DIR argument_quality_ranking DIR big_patent_innovation 2 files DIR boardgame_qa 2 files DIR chatbot_arena_conversations 2 files DIR civil_comments 2 files DIR code_judge_bench DIR colbert_humor_detection 2 files DIR customer_support_tickets_en 2 files DIR customer_support_tickets_gorkem 2 files DIR dbpedia_easy 2 files DIR dbpedia_hard 2 files DIR dbpedia_medium 2 files DIR enron_email_quality 2 files DIR enron_email_type 2 files DIR enron_reply_quality 2 files DIR go_emotions 2 files DIR gpqa_diamond DIR halueval_dialogue 2 files DIR halueval_qa 2 files DIR halueval_summarization DIR hh_rlhf 2 files DIR judge_bench DIR lex_glue_case_hold 2 files DIR lex_glue_ledgar 2 files DIR medical_abstracts 2 files DIR mfrc 2 files DIR mmlu 2 files DIR mmlu_pro DIR mt_bench_human_judgments DIR musr_murder_mysteries DIR musr_object_placements DIR musr_team_allocation DIR or_bench_80k 2 files DIR or_bench_hard_1k 2 files DIR or_bench_toxic DIR projudgebench 2 files DIR reward_bench_2 2 files DIR rod101_essay_scoring DIR sem_eval_2010_task_8 2 files DIR smollm_corpus 2 files DIR snli 2 files DIR spartqa_mchoice 2 files DIR toxigen_data 2 files DIR tweet_eval_emotion 2 files DIR tweet_eval_hate 2 files DIR tweet_eval_irony 2 files DIR tweet_eval_offensive 2 files DIR tweet_eval_sentiment 2 files DIR ultrafeedback 2 files DIR writingprompts_quality 2 files DIR yahoo_answers_quality 2 files DIR yelp 2 files FILE .gitattributes FILE README.md

File contents live on Hugging Face. Click a file or folder to open it there.

Download the dataset

0

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python) 
from datasets import load_dataset
ds = load_dataset("tytodd/qwen3.5-2b-v2-smoke")
Raw snapshot (Python) 
from huggingface_hub import snapshot_download
snapshot_download(repo_id="tytodd/qwen3.5-2b-v2-smoke", repo_type="dataset")
git clone (requires git-lfs) 
git clone https://huggingface.co/datasets/tytodd/qwen3.5-2b-v2-smoke
Browse repository on Hugging Face ->
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Get the app Browse the catalog
Open in Abliteration