pretty_name: Evaluation run of zetasepic/Qwen2.5-32B-Instruct-abliterated-v2
dataset_summary: "Dataset automatically created during the evaluation run of model
\ zetasepic/Qwen2.5-32B-Instruct-abliterated-v2.\n
\nThe dataset is composed of 116 configuration, each one coresponding to one of
\ the evaluated task.\n\nThe dataset has been created from 1 run(s). Each run can
\ be found as a specific split in each configuration, the split being named using
\ the timestamp of the run.The "train" split is always pointing to the latest
\ results.\n\nAn additional configuration "results" store all the aggregated results
\ of the run.\n\nTo load the details from a run, you can for instance do the following:\n\
,\n\t\"results\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\nThese are the\
\ [latest results from run 2025-03-08T00:19:38.075806](https://huggingface.co/datasets/OALL/details_zetasepic__Qwen2.5-32B-Instruct-abliterated-v2_v2/blob/main/results_2025-03-08T00-19-38.075806.json)(note\
\ that their might be results for other tasks in the repos if successive evals didn't\
\ cover the same tasks. You find each in the results and the \"latest\" split for\
\ each eval):\n\n```python\n{\n \"all\": {\n \"acc_norm\": 0.692010832789437,\n\
\ \"acc_norm_stderr\": 0.030785943969471326\n },\n \"community|alghafa:meta_ar_dialects|0\"\
: {\n \"acc_norm\": 0.8163113994439296,\n \"acc_norm_stderr\": 0.005272463262167873\n\
\ },\n \"community|alghafa:meta_ar_msa|0\": {\n \"acc_norm\": 0.9083798882681564,\n\
\ \"acc_norm_stderr\": 0.009648513398613853\n },\n \"community|alghafa:multiple_choice_facts_truefalse_balanced_task|0\"\
: {\n \"acc_norm\": 0.9466666666666667,\n \"acc_norm_stderr\": 0.02612053372889386\n\
\ },\n \"community|alghafa:multiple_choice_grounded_statement_soqal_task|0\"\
: {\n \"acc_norm\": 0.9266666666666666,\n \"acc_norm_stderr\": 0.021355972327761853\n\
\ },\n \"community|alghafa:multiple_choice_grounded_statement_xglue_mlqa_task|0\"\
: {\n \"acc_norm\": 0.9066666666666666,\n \"acc_norm_stderr\": 0.023831373802832918\n\
\ },\n \"community|alghafa:multiple_choice_rating_sentiment_no_neutral_task|0\"\
: {\n \"acc_norm\": 0.7931207004377736,\n \"acc_norm_stderr\": 0.004530499524480054\n\
\ },\n \"community|alghafa:multiple_choice_rating_sentiment_task|0\": {\n\
\ \"acc_norm\": 0.5461217681401168,\n \"acc_norm_stderr\": 0.006430667453785393\n\
\ },\n \"community|alghafa:multiple_choice_sentiment_task|0\": {\n \
\ \"acc_norm\": 0.39476744186046514,\n \"acc_norm_stderr\": 0.011789458733769523\n\
\ },\n \"community|arabic_exams|0\": {\n \"acc_norm\": 0.5586592178770949,\n\
\ \"acc_norm_stderr\": 0.02144757127806544\n },\n \"community|arabic_mmlu:Accounting\
\ (University)|0\": {\n \"acc_norm\": 0.7027027027027027,\n \"acc_norm_stderr\"\
: 0.053495821667206706\n },\n \"community|arabic_mmlu:Arabic Language (General)|0\"\
: {\n \"acc_norm\": 0.6062091503267973,\n \"acc_norm_stderr\": 0.019766211991073077\n\
\ },\n \"community|arabic_mmlu:Arabic Language (Grammar)|0\": {\n \"\
acc_norm\": 0.4191780821917808,\n \"acc_norm_stderr\": 0.02586247534120004\n\
\ },\n \"community|arabic_mmlu:Arabic Language (High School)|0\": {\n \
\ \"acc_norm\": 0.46153846153846156,\n \"acc_norm_stderr\": 0.025275892070240634\n\
\ },\n \"community|arabic_mmlu:Arabic Language (Middle School)|0\": {\n \
\ \"acc_norm\": 0.7407407407407407,\n \"acc_norm_stderr\": 0.08594360757264022\n\
\ },\n \"community|arabic_mmlu:Arabic Language (Primary School)|0\": {\n \
\ \"acc_norm\": 0.7619047619047619,\n \"acc_norm_stderr\": 0.026883687473220855\n\
\ },\n \"community|arabic_mmlu:Biology (High School)|0\": {\n \"acc_norm\"\
: 0.5585521646557843,\n \"acc_norm_stderr\": 0.0132333633767092\n },\n\
\ \"community|arabic_mmlu:Civics (High School)|0\": {\n \"acc_norm\":\
\ 0.4942528735632184,\n \"acc_norm_stderr\": 0.053912824825556656\n },\n\
\ \"community|arabic_mmlu:Civics (Middle School)|0\": {\n \"acc_norm\"\
: 0.597457627118644,\n \"acc_norm_stderr\": 0.03199082387826596\n },\n\
\ \"community|arabic_mmlu:Computer Science (High School)|0\": {\n \"acc_norm\"\
: 0.7241379310344828,\n \"acc_norm_stderr\": 0.027718515602362118\n },\n\
\ \"community|arabic_mmlu:Computer Science (Middle School)|0\": {\n \"\
acc_norm\": 0.9259259259259259,\n \"acc_norm_stderr\": 0.051361129280113806\n\
\ },\n \"community|arabic_mmlu:Computer Science (Primary School)|0\": {\n\
\ \"acc_norm\": 0.7631578947368421,\n \"acc_norm_stderr\": 0.03092471849436861\n\
\ },\n \"community|arabic_mmlu:Computer Science (University)|0\": {\n \
\ \"acc_norm\": 0.78125,\n \"acc_norm_stderr\": 0.052083333333333336\n\
\ },\n \"community|arabic_mmlu:Driving Test|0\": {\n \"acc_norm\":\
\ 0.7332782824112304,\n \"acc_norm_stderr\": 0.012713669449538135\n },\n\
\ \"community|arabic_mmlu:Economics (High School)|0\": {\n \"acc_norm\"\
: 0.7444444444444445,\n \"acc_norm_stderr\": 0.023020332152774988\n },\n\
\ \"community|arabic_mmlu:Economics (Middle School)|0\": {\n \"acc_norm\"\
: 0.8160919540229885,\n \"acc_norm_stderr\": 0.04177540678018986\n },\n\
\ \"community|arabic_mmlu:Economics (University)|0\": {\n \"acc_norm\"\
: 0.635036496350365,\n \"acc_norm_stderr\": 0.041281418039994466\n },\n\
\ \"community|arabic_mmlu:General Knowledge|0\": {\n \"acc_norm\": 0.6898148148148148,\n\
\ \"acc_norm_stderr\": 0.015746041312239174\n },\n \"community|arabic_mmlu:General\
\ Knowledge (Middle School)|0\": {\n \"acc_norm\": 0.7674418604651163,\n\
\ \"acc_norm_stderr\": 0.03230654083203451\n },\n \"community|arabic_mmlu:General\
\ Knowledge (Primary School)|0\": {\n \"acc_norm\": 0.7530864197530864,\n\
\ \"acc_norm_stderr\": 0.03398458402215315\n },\n \"community|arabic_mmlu:Geography\
\ (High School)|0\": {\n \"acc_norm\": 0.6059730250481695,\n \"acc_norm_stderr\"\
: 0.015174005775184334\n },\n \"community|arabic_mmlu:Geography (Middle School)|0\"\
: {\n \"acc_norm\": 0.7426470588235294,\n \"acc_norm_stderr\": 0.026556519470041517\n\
\ },\n \"community|arabic_mmlu:Geography (Primary School)|0\": {\n \
\ \"acc_norm\": 0.7543859649122807,\n \"acc_norm_stderr\": 0.05752139056405265\n\
\ },\n \"community|arabic_mmlu:History (High School)|0\": {\n \"acc_norm\"\
: 0.4776315789473684,\n \"acc_norm_stderr\": 0.018130679701241187\n },\n\
\ \"community|arabic_mmlu:History (Middle School)|0\": {\n \"acc_norm\"\
: 0.6650246305418719,\n \"acc_norm_stderr\": 0.033208527423483104\n },\n\
\ \"community|arabic_mmlu:History (Primary School)|0\": {\n \"acc_norm\"\
: 0.6862745098039216,\n \"acc_norm_stderr\": 0.04617034827006719\n },\n\
\ \"community|arabic_mmlu:Islamic Studies|0\": {\n \"acc_norm\": 0.568075117370892,\n\
\ \"acc_norm_stderr\": 0.019610860755030473\n },\n \"community|arabic_mmlu:Islamic\
\ Studies (High School)|0\": {\n \"acc_norm\": 0.7335329341317365,\n \
\ \"acc_norm_stderr\": 0.024227547959039863\n },\n \"community|arabic_mmlu:Islamic\
\ Studies (Middle School)|0\": {\n \"acc_norm\": 0.6974789915966386,\n \
\ \"acc_norm_stderr\": 0.02983796238829194\n },\n \"community|arabic_mmlu:Islamic\
\ Studies (Primary School)|0\": {\n \"acc_norm\": 0.8068068068068068,\n \
\ \"acc_norm_stderr\": 0.012497277591195375\n },\n \"community|arabic_mmlu:Law\
\ (Professional)|0\": {\n \"acc_norm\": 0.767515923566879,\n \"acc_norm_stderr\"\
: 0.023876360884096257\n },\n \"community|arabic_mmlu:Management (University)|0\"\
: {\n \"acc_norm\": 0.8266666666666667,\n \"acc_norm_stderr\": 0.04400382183783964\n\
\ },\n \"community|arabic_mmlu:Math (Primary School)|0\": {\n \"acc_norm\"\
: 0.78239608801956,\n \"acc_norm_stderr\": 0.020427565086683957\n },\n\
\ \"community|arabic_mmlu:Natural Science (Middle School)|0\": {\n \"\
acc_norm\": 0.8347107438016529,\n \"acc_norm_stderr\": 0.023926644290131693\n\
\ },\n \"community|arabic_mmlu:Natural Science (Primary School)|0\": {\n \
\ \"acc_norm\": 0.8869047619047619,\n \"acc_norm_stderr\": 0.017303671012065352\n\
\ },\n \"community|arabic_mmlu:Philosophy (High School)|0\": {\n \"\
acc_norm\": 0.6923076923076923,\n \"acc_norm_stderr\": 0.07487142513727499\n\
\ },\n \"community|arabic_mmlu:Physics (High School)|0\": {\n \"acc_norm\"\
: 0.6039215686274509,\n \"acc_norm_stderr\": 0.03068767658826368\n },\n\
\ \"community|arabic_mmlu:Political Science (University)|0\": {\n \"acc_norm\"\
: 0.6904761904761905,\n \"acc_norm_stderr\": 0.03197777494209474\n },\n\
\ \"community|arabic_mmlu:Social Science (Middle School)|0\": {\n \"acc_norm\"\
: 0.5477178423236515,\n \"acc_norm_stderr\": 0.03212754578265449\n },\n\
\ \"community|arabic_mmlu:Social Science (Primary School)|0\": {\n \"\
acc_norm\": 0.7843971631205674,\n \"acc_norm_stderr\": 0.015499186819256007\n\
\ },\n \"community|arabic_mmlu_ht:abstract_algebra|0\": {\n \"acc_norm\"\
: 0.38,\n \"acc_norm_stderr\": 0.048783173121456316\n },\n \"community|arabic_mmlu_ht:anatomy|0\"\
: {\n \"acc_norm\": 0.5185185185185185,\n \"acc_norm_stderr\": 0.043163785995113245\n\
\ },\n \"community|arabic_mmlu_ht:astronomy|0\": {\n \"acc_norm\":\
\ 0.7894736842105263,\n \"acc_norm_stderr\": 0.03317672787533157\n },\n\
\ \"community|arabic_mmlu_ht:business_ethics|0\": {\n \"acc_norm\": 0.68,\n\
\ \"acc_norm_stderr\": 0.04688261722621504\n },\n \"community|arabic_mmlu_ht:clinical_knowledge|0\"\
: {\n \"acc_norm\": 0.660377358490566,\n \"acc_norm_stderr\": 0.02914690474779832\n\
\ },\n \"community|arabic_mmlu_ht:college_biology|0\": {\n \"acc_norm\"\
: 0.6875,\n \"acc_norm_stderr\": 0.038760854559127644\n },\n \"community|arabic_mmlu_ht:college_chemistry|0\"\
: {\n \"acc_norm\": 0.44,\n \"acc_norm_stderr\": 0.049888765156985884\n\
\ },\n \"community|arabic_mmlu_ht:college_computer_science|0\": {\n \
\ \"acc_norm\": 0.63,\n \"acc_norm_stderr\": 0.04852365870939099\n },\n\
\ \"community|arabic_mmlu_ht:college_mathematics|0\": {\n \"acc_norm\"\
: 0.48,\n \"acc_norm_stderr\": 0.050211673156867795\n },\n \"community|arabic_mmlu_ht:college_medicine|0\"\
: {\n \"acc_norm\": 0.6011560693641619,\n \"acc_norm_stderr\": 0.0373362665538351\n\
\ },\n \"community|arabic_mmlu_ht:college_physics|0\": {\n \"acc_norm\"\
: 0.5098039215686274,\n \"acc_norm_stderr\": 0.04974229460422817\n },\n\
\ \"community|arabic_mmlu_ht:computer_security|0\": {\n \"acc_norm\":\
\ 0.73,\n \"acc_norm_stderr\": 0.044619604333847394\n },\n \"community|arabic_mmlu_ht:conceptual_physics|0\"\
: {\n \"acc_norm\": 0.7106382978723405,\n \"acc_norm_stderr\": 0.029644006577009618\n\
\ },\n \"community|arabic_mmlu_ht:econometrics|0\": {\n \"acc_norm\"\
: 0.47368421052631576,\n \"acc_norm_stderr\": 0.046970851366478626\n },\n\
\ \"community|arabic_mmlu_ht:electrical_engineering|0\": {\n \"acc_norm\"\
: 0.5862068965517241,\n \"acc_norm_stderr\": 0.04104269211806232\n },\n\
\ \"community|arabic_mmlu_ht:elementary_mathematics|0\": {\n \"acc_norm\"\
: 0.6746031746031746,\n \"acc_norm_stderr\": 0.024130158299762613\n },\n\
\ \"community|arabic_mmlu_ht:formal_logic|0\": {\n \"acc_norm\": 0.5714285714285714,\n\
\ \"acc_norm_stderr\": 0.0442626668137991\n },\n \"community|arabic_mmlu_ht:global_facts|0\"\
: {\n \"acc_norm\": 0.42,\n \"acc_norm_stderr\": 0.049604496374885836\n\
\ },\n \"community|arabic_mmlu_ht:high_school_biology|0\": {\n \"acc_norm\"\
: 0.7175452399685287,\n \"acc_norm_stderr\": 0.007291596820938285\n },\n\
\ \"community|arabic_mmlu_ht:high_school_chemistry|0\": {\n \"acc_norm\"\
: 0.7158864541832669,\n \"acc_norm_stderr\": 0.007117463813276186\n },\n\
\ \"community|arabic_mmlu_ht:high_school_computer_science|0\": {\n \"\
acc_norm\": 0.8,\n \"acc_norm_stderr\": 0.04020151261036846\n },\n \
\ \"community|arabic_mmlu_ht:high_school_european_history|0\": {\n \"acc_norm\"\
: 0.5976447497546614,\n \"acc_norm_stderr\": 0.005431514510405362\n },\n\
\ \"community|arabic_mmlu_ht:high_school_geography|0\": {\n \"acc_norm\"\
: 0.803030303030303,\n \"acc_norm_stderr\": 0.028335609732463362\n },\n\
\ \"community|arabic_mmlu_ht:high_school_government_and_politics|0\": {\n \
\ \"acc_norm\": 0.8082901554404145,\n \"acc_norm_stderr\": 0.028408953626245282\n\
\ },\n \"community|arabic_mmlu_ht:high_school_macroeconomics|0\": {\n \
\ \"acc_norm\": 0.7295053614666206,\n \"acc_norm_stderr\": 0.008263131293548422\n\
\ },\n \"community|arabic_mmlu_ht:high_school_mathematics|0\": {\n \
\ \"acc_norm\": 0.42962962962962964,\n \"acc_norm_stderr\": 0.030182099804387262\n\
\ },\n \"community|arabic_mmlu_ht:high_school_microeconomics|0\": {\n \
\ \"acc_norm\": 0.7184873949579832,\n \"acc_norm_stderr\": 0.02921354941437216\n\
\ },\n \"community|arabic_mmlu_ht:high_school_physics|0\": {\n \"acc_norm\"\
: 0.5033112582781457,\n \"acc_norm_stderr\": 0.04082393379449654\n },\n\
\ \"community|arabic_mmlu_ht:high_school_psychology|0\": {\n \"acc_norm\"\
: 0.7651376146788991,\n \"acc_norm_stderr\": 0.018175110510343578\n },\n\
\ \"community|arabic_mmlu_ht:high_school_statistics|0\": {\n \"acc_norm\"\
: 0.7164461247637051,\n \"acc_norm_stderr\": 0.0069292790274951555\n },\n\
\ \"community|arabic_mmlu_ht:high_school_us_history|0\": {\n \"acc_norm\"\
: 0.7745098039215687,\n \"acc_norm_stderr\": 0.029331162294251742\n },\n\
\ \"community|arabic_mmlu_ht:high_school_world_history|0\": {\n \"acc_norm\"\
: 0.8396624472573839,\n \"acc_norm_stderr\": 0.02388438092596567\n },\n\
\ \"community|arabic_mmlu_ht:human_aging|0\": {\n \"acc_norm\": 0.672645739910314,\n\
\ \"acc_norm_stderr\": 0.03149384670994131\n },\n \"community|arabic_mmlu_ht:human_sexuality|0\"\
: {\n \"acc_norm\": 0.6335877862595419,\n \"acc_norm_stderr\": 0.04225875451969637\n\
\ },\n \"community|arabic_mmlu_ht:international_law|0\": {\n \"acc_norm\"\
: 0.8016528925619835,\n \"acc_norm_stderr\": 0.036401182719909476\n },\n\
\ \"community|arabic_mmlu_ht:jurisprudence|0\": {\n \"acc_norm\": 0.7314814814814815,\n\
\ \"acc_norm_stderr\": 0.042844679680521934\n },\n \"community|arabic_mmlu_ht:logical_fallacies|0\"\
: {\n \"acc_norm\": 0.6871165644171779,\n \"acc_norm_stderr\": 0.03642914578292406\n\
\ },\n \"community|arabic_mmlu_ht:machine_learning|0\": {\n \"acc_norm\"\
: 0.48214285714285715,\n \"acc_norm_stderr\": 0.047427623612430116\n },\n\
\ \"community|arabic_mmlu_ht:management|0\": {\n \"acc_norm\": 0.6796116504854369,\n\
\ \"acc_norm_stderr\": 0.04620284082280041\n },\n \"community|arabic_mmlu_ht:marketing|0\"\
: {\n \"acc_norm\": 0.8034188034188035,\n \"acc_norm_stderr\": 0.02603538609895129\n\
\ },\n \"community|arabic_mmlu_ht:medical_genetics|0\": {\n \"acc_norm\"\
: 0.63,\n \"acc_norm_stderr\": 0.04852365870939099\n },\n \"community|arabic_mmlu_ht:miscellaneous|0\"\
: {\n \"acc_norm\": 0.743801652892562,\n \"acc_norm_stderr\": 0.008875621982917577\n\
\ },\n \"community|arabic_mmlu_ht:moral_disputes|0\": {\n \"acc_norm\"\
: 0.6820809248554913,\n \"acc_norm_stderr\": 0.025070713719153186\n },\n\
\ \"community|arabic_mmlu_ht:moral_scenarios|0\": {\n \"acc_norm\": 0.506145251396648,\n\
\ \"acc_norm_stderr\": 0.01672123848363141\n },\n \"community|arabic_mmlu_ht:nutrition|0\"\
: {\n \"acc_norm\": 0.7222222222222222,\n \"acc_norm_stderr\": 0.025646863097137904\n\
\ },\n \"community|arabic_mmlu_ht:philosophy|0\": {\n \"acc_norm\"\
: 0.707395498392283,\n \"acc_norm_stderr\": 0.025839898334877976\n },\n\
\ \"community|arabic_mmlu_ht:prehistory|0\": {\n \"acc_norm\": 0.6728395061728395,\n\
\ \"acc_norm_stderr\": 0.02610567386140983\n },\n \"community|arabic_mmlu_ht:professional_accounting|0\"\
: {\n \"acc_norm\": 0.7002658396101019,\n \"acc_norm_stderr\": 0.006819725978102662\n\
\ },\n \"community|arabic_mmlu_ht:professional_law|0\": {\n \"acc_norm\"\
: 0.5940903968949544,\n \"acc_norm_stderr\": 0.005495108555167751\n },\n\
\ \"community|arabic_mmlu_ht:professional_medicine|0\": {\n \"acc_norm\"\
: 0.7354917532070862,\n \"acc_norm_stderr\": 0.010904778852241306\n },\n\
\ \"community|arabic_mmlu_ht:professional_psychology|0\": {\n \"acc_norm\"\
: 0.7116757065372538,\n \"acc_norm_stderr\": 0.0076546196855586295\n },\n\
\ \"community|arabic_mmlu_ht:public_relations|0\": {\n \"acc_norm\": 0.5818181818181818,\n\
\ \"acc_norm_stderr\": 0.04724577405731573\n },\n \"community|arabic_mmlu_ht:security_studies|0\"\
: {\n \"acc_norm\": 0.7142857142857143,\n \"acc_norm_stderr\": 0.028920583220675596\n\
\ },\n \"community|arabic_mmlu_ht:sociology|0\": {\n \"acc_norm\":\
\ 0.746268656716418,\n \"acc_norm_stderr\": 0.03076944496729602\n },\n\
\ \"community|arabic_mmlu_ht:us_foreign_policy|0\": {\n \"acc_norm\":\
\ 0.82,\n \"acc_norm_stderr\": 0.038612291966536934\n },\n \"community|arabic_mmlu_ht:virology|0\"\
: {\n \"acc_norm\": 0.5301204819277109,\n \"acc_norm_stderr\": 0.03885425420866767\n\
\ },\n \"community|arabic_mmlu_ht:world_religions|0\": {\n \"acc_norm\"\
: 0.7134502923976608,\n \"acc_norm_stderr\": 0.03467826685703826\n },\n\
\ \"community|aratrust:Ethics|0\": {\n \"acc_norm\": 0.8166666666666667,\n\
\ \"acc_norm_stderr\": 0.05037523978531017\n },\n \"community|aratrust:Illegal|0\"\
: {\n \"acc_norm\": 0.9056603773584906,\n \"acc_norm_stderr\": 0.040534804190911294\n\
\ },\n \"community|aratrust:MentalHealth|0\": {\n \"acc_norm\": 0.9210526315789473,\n\
\ \"acc_norm_stderr\": 0.031137262016313737\n },\n \"community|aratrust:Offensive|0\"\
: {\n \"acc_norm\": 0.855072463768116,\n \"acc_norm_stderr\": 0.042689636687312106\n\
\ },\n \"community|aratrust:PhysicalHealth|0\": {\n \"acc_norm\": 0.9178082191780822,\n\
\ \"acc_norm_stderr\": 0.03236860017981242\n },\n \"community|aratrust:Privacy|0\"\
: {\n \"acc_norm\": 0.9649122807017544,\n \"acc_norm_stderr\": 0.024588241692166144\n\
\ },\n \"community|aratrust:Trustfulness|0\": {\n \"acc_norm\": 0.782051282051282,\n\
\ \"acc_norm_stderr\": 0.047048893371093484\n },\n \"community|aratrust:Unfairness|0\"\
: {\n \"acc_norm\": 0.9090909090909091,\n \"acc_norm_stderr\": 0.03912104390108503\n\
\ },\n \"community|madinah_qa:Arabic Language (General)|0\": {\n \"\
acc_norm\": 0.7941176470588235,\n \"acc_norm_stderr\": 0.01635804429747852\n\
\ },\n \"community|madinah_qa:Arabic Language (Grammar)|0\": {\n \"\
acc_norm\": 0.31232876712328766,\n \"acc_norm_stderr\": 0.024291018800569122\n\
\ },\n \"community|alghafa:_average|0\": {\n \"acc_norm\": 0.7798376497688053,\n\
\ \"acc_norm_stderr\": 0.013622435279038166\n },\n \"community|arabic_mmlu:_average|0\"\
: {\n \"acc_norm\": 0.6957761961875122,\n \"acc_norm_stderr\": 0.0324229297443301\n\
\ },\n \"community|arabic_mmlu_ht:_average|0\": {\n \"acc_norm\": 0.6572997736047431,\n\
\ \"acc_norm_stderr\": 0.03149671003952715\n },\n \"community|aratrust:_average|0\"\
: {\n \"acc_norm\": 0.884039353799281,\n \"acc_norm_stderr\": 0.03848296522800054\n\
\ },\n \"community|madinah_qa:_average|0\": {\n \"acc_norm\": 0.5532232070910555,\n\
\ \"acc_norm_stderr\": 0.02032453154902382\n }\n}\n```"
repo_url: https://huggingface.co/zetasepic/Qwen2.5-32B-Instruct-abliterated-v2
configs:
- config_name: community_alghafa_meta_ar_dialects_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:meta_ar_dialects|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:meta_ar_dialects|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_meta_ar_msa_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:meta_ar_msa|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:meta_ar_msa|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_multiple_choice_facts_truefalse_balanced_task_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:multiple_choice_facts_truefalse_balanced_task|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:multiple_choice_facts_truefalse_balanced_task|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_multiple_choice_grounded_statement_soqal_task_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:multiple_choice_grounded_statement_soqal_task|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:multiple_choice_grounded_statement_soqal_task|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:multiple_choice_grounded_statement_xglue_mlqa_task|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:multiple_choice_grounded_statement_xglue_mlqa_task|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_multiple_choice_rating_sentiment_no_neutral_task_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:multiple_choice_rating_sentiment_no_neutral_task|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:multiple_choice_rating_sentiment_no_neutral_task|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_multiple_choice_rating_sentiment_task_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:multiple_choice_rating_sentiment_task|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:multiple_choice_rating_sentiment_task|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_alghafa_multiple_choice_sentiment_task_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|alghafa:multiple_choice_sentiment_task|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|alghafa:multiple_choice_sentiment_task|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_exams_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_exams|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_exams|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Accounting__University__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Accounting (University)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Accounting (University)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Arabic_Language__General__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Arabic Language (General)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Arabic Language (General)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Arabic_Language__Grammar__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Arabic Language (Grammar)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Arabic Language (Grammar)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Arabic_Language__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Arabic Language (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Arabic Language (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Arabic_Language__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Arabic Language (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Arabic Language (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Arabic_Language__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Arabic Language (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Arabic Language (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Biology__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Biology (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Biology (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Civics__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Civics (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Civics (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Civics__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Civics (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Civics (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Computer_Science__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Computer Science (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Computer Science (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Computer_Science__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Computer Science (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Computer Science (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Computer_Science__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Computer Science (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Computer Science (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Computer_Science__University__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Computer Science (University)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Computer Science (University)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Driving_Test_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Driving Test|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Driving Test|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Economics__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Economics (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Economics (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Economics__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Economics (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Economics (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Economics__University__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Economics (University)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Economics (University)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_General_Knowledge_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:General Knowledge|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:General Knowledge|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_General_Knowledge__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:General Knowledge (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:General Knowledge (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_General_Knowledge__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:General Knowledge (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:General Knowledge (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Geography__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Geography (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Geography (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Geography__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Geography (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Geography (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Geography__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Geography (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Geography (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_History__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:History (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:History (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_History__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:History (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:History (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_History__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:History (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:History (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Islamic_Studies_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Islamic Studies|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Islamic Studies|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Islamic_Studies__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Islamic Studies (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Islamic Studies (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Islamic_Studies__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Islamic Studies (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Islamic Studies (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Islamic_Studies__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Islamic Studies (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Islamic Studies (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Law__Professional__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Law (Professional)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Law (Professional)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Management__University__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Management (University)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Management (University)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Math__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Math (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Math (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Natural_Science__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Natural Science (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Natural Science (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Natural_Science__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Natural Science (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Natural Science (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Philosophy__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Philosophy (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Philosophy (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Physics__High_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Physics (High School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Physics (High School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Political_Science__University__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Political Science (University)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Political Science (University)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Social_Science__Middle_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Social Science (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Social Science (Middle School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_Social_Science__Primary_School__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu:Social Science (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu:Social Science (Primary School)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_abstract_algebra_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:abstract_algebra|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:abstract_algebra|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_anatomy_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:anatomy|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:anatomy|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_astronomy_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:astronomy|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:astronomy|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_business_ethics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:business_ethics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:business_ethics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_clinical_knowledge_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:clinical_knowledge|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:clinical_knowledge|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_college_biology_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:college_biology|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:college_biology|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_college_chemistry_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:college_chemistry|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:college_chemistry|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_college_computer_science_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:college_computer_science|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:college_computer_science|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_college_mathematics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:college_mathematics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:college_mathematics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_college_medicine_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:college_medicine|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:college_medicine|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_college_physics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:college_physics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:college_physics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_computer_security_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:computer_security|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:computer_security|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_conceptual_physics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:conceptual_physics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:conceptual_physics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_econometrics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:econometrics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:econometrics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_electrical_engineering_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:electrical_engineering|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:electrical_engineering|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_elementary_mathematics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:elementary_mathematics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:elementary_mathematics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_formal_logic_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:formal_logic|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:formal_logic|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_global_facts_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:global_facts|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:global_facts|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_biology_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_biology|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_biology|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_chemistry_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_chemistry|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_chemistry|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_computer_science_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_computer_science|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_computer_science|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_european_history_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_european_history|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_european_history|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_geography_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_geography|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_geography|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_government_and_politics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_government_and_politics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_government_and_politics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_macroeconomics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_macroeconomics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_macroeconomics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_mathematics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_mathematics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_mathematics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_microeconomics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_microeconomics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_microeconomics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_physics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_physics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_physics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_psychology_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_psychology|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_psychology|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_statistics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_statistics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_statistics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_us_history_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_us_history|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_us_history|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_high_school_world_history_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:high_school_world_history|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:high_school_world_history|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_human_aging_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:human_aging|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:human_aging|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_human_sexuality_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:human_sexuality|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:human_sexuality|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_international_law_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:international_law|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:international_law|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_jurisprudence_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:jurisprudence|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:jurisprudence|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_logical_fallacies_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:logical_fallacies|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:logical_fallacies|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_machine_learning_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:machine_learning|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:machine_learning|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_management_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:management|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:management|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_marketing_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:marketing|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:marketing|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_medical_genetics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:medical_genetics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:medical_genetics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_miscellaneous_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:miscellaneous|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:miscellaneous|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_moral_disputes_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:moral_disputes|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:moral_disputes|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_moral_scenarios_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:moral_scenarios|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:moral_scenarios|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_nutrition_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:nutrition|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:nutrition|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_philosophy_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:philosophy|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:philosophy|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_prehistory_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:prehistory|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:prehistory|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_professional_accounting_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:professional_accounting|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:professional_accounting|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_professional_law_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:professional_law|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:professional_law|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_professional_medicine_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:professional_medicine|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:professional_medicine|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_professional_psychology_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:professional_psychology|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:professional_psychology|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_public_relations_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:public_relations|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:public_relations|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_security_studies_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:security_studies|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:security_studies|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_sociology_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:sociology|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:sociology|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_us_foreign_policy_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:us_foreign_policy|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:us_foreign_policy|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_virology_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:virology|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:virology|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_arabic_mmlu_ht_world_religions_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|arabic_mmlu_ht:world_religions|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|arabic_mmlu_ht:world_religions|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_Ethics_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:Ethics|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:Ethics|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_Illegal_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:Illegal|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:Illegal|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_MentalHealth_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:MentalHealth|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:MentalHealth|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_Offensive_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:Offensive|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:Offensive|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_PhysicalHealth_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:PhysicalHealth|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:PhysicalHealth|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_Privacy_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:Privacy|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:Privacy|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_Trustfulness_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:Trustfulness|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:Trustfulness|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_aratrust_Unfairness_0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|aratrust:Unfairness|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|aratrust:Unfairness|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_madinah_qa_Arabic_Language__General__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|madinah_qa:Arabic Language (General)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|madinah_qa:Arabic Language (General)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: community_madinah_qa_Arabic_Language__Grammar__0
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- '**/details_community|madinah_qa:Arabic Language (Grammar)|0_2025-03-08T00-19-38.075806.parquet'
- split: latest
path:
- '**/details_community|madinah_qa:Arabic Language (Grammar)|0_2025-03-08T00-19-38.075806.parquet'
- config_name: results
data_files:
- split: 2025_03_08T00_19_38.075806
path:
- results_2025-03-08T00-19-38.075806.parquet
- split: latest
path:
- results_2025-03-08T00-19-38.075806.parquet
---
# Dataset Card for Evaluation run of zetasepic/Qwen2.5-32B-Instruct-abliterated-v2
<!-- Provide a quick summary of the dataset. -->
Dataset automatically created during the evaluation run of model [zetasepic/Qwen2.5-32B-Instruct-abliterated-v2](https://huggingface.co/zetasepic/Qwen2.5-32B-Instruct-abliterated-v2).
The dataset is composed of 116 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
An additional configuration "results" store all the aggregated results of the run.
To load the details from a run, you can for instance do the following:
```python
from datasets import load_dataset
data = load_dataset("OALL/details_zetasepic__Qwen2.5-32B-Instruct-abliterated-v2_v2",
"results",
split="train")
Latest results
These are the latest results from run 2025-03-08T00:19:38.075806(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval):
{
"all": {
"acc_norm": 0.692010832789437,
"acc_norm_stderr": 0.030785943969471326
},
"community|alghafa:meta_ar_dialects|0": {
"acc_norm": 0.8163113994439296,
"acc_norm_stderr": 0.005272463262167873
},
"community|alghafa:meta_ar_msa|0": {
"acc_norm": 0.9083798882681564,
"acc_norm_stderr": 0.009648513398613853
},
"community|alghafa:multiple_choice_facts_truefalse_balanced_task|0": {
"acc_norm": 0.9466666666666667,
"acc_norm_stderr": 0.02612053372889386
},
"community|alghafa:multiple_choice_grounded_statement_soqal_task|0": {
"acc_norm": 0.9266666666666666,
"acc_norm_stderr": 0.021355972327761853
},
"community|alghafa:multiple_choice_grounded_statement_xglue_mlqa_task|0": {
"acc_norm": 0.9066666666666666,
"acc_norm_stderr": 0.023831373802832918
},
"community|alghafa:multiple_choice_rating_sentiment_no_neutral_task|0": {
"acc_norm": 0.7931207004377736,
"acc_norm_stderr": 0.004530499524480054
},
"community|alghafa:multiple_choice_rating_sentiment_task|0": {
"acc_norm": 0.5461217681401168,
"acc_norm_stderr": 0.006430667453785393
},
"community|alghafa:multiple_choice_sentiment_task|0": {
"acc_norm": 0.39476744186046514,
"acc_norm_stderr": 0.011789458733769523
},
"community|arabic_exams|0": {
"acc_norm": 0.5586592178770949,
"acc_norm_stderr": 0.02144757127806544
},
"community|arabic_mmlu:Accounting (University)|0": {
"acc_norm": 0.7027027027027027,
"acc_norm_stderr": 0.053495821667206706
},
"community|arabic_mmlu:Arabic Language (General)|0": {
"acc_norm": 0.6062091503267973,
"acc_norm_stderr": 0.019766211991073077
},
"community|arabic_mmlu:Arabic Language (Grammar)|0": {
"acc_norm": 0.4191780821917808,
"acc_norm_stderr": 0.02586247534120004
},
"community|arabic_mmlu:Arabic Language (High School)|0": {
"acc_norm": 0.46153846153846156,
"acc_norm_stderr": 0.025275892070240634
},
"community|arabic_mmlu:Arabic Language (Middle School)|0": {
"acc_norm": 0.7407407407407407,
"acc_norm_stderr": 0.08594360757264022
},
"community|arabic_mmlu:Arabic Language (Primary School)|0": {
"acc_norm": 0.7619047619047619,
"acc_norm_stderr": 0.026883687473220855
},
"community|arabic_mmlu:Biology (High School)|0": {
"acc_norm": 0.5585521646557843,
"acc_norm_stderr": 0.0132333633767092
},
"community|arabic_mmlu:Civics (High School)|0": {
"acc_norm": 0.4942528735632184,
"acc_norm_stderr": 0.053912824825556656
},
"community|arabic_mmlu:Civics (Middle School)|0": {
"acc_norm": 0.597457627118644,
"acc_norm_stderr": 0.03199082387826596
},
"community|arabic_mmlu:Computer Science (High School)|0": {
"acc_norm": 0.7241379310344828,
"acc_norm_stderr": 0.027718515602362118
},
"community|arabic_mmlu:Computer Science (Middle School)|0": {
"acc_norm": 0.9259259259259259,
"acc_norm_stderr": 0.051361129280113806
},
"community|arabic_mmlu:Computer Science (Primary School)|0": {
"acc_norm": 0.7631578947368421,
"acc_norm_stderr": 0.03092471849436861
},
"community|arabic_mmlu:Computer Science (University)|0": {
"acc_norm": 0.78125,
"acc_norm_stderr": 0.052083333333333336
},
"community|arabic_mmlu:Driving Test|0": {
"acc_norm": 0.7332782824112304,
"acc_norm_stderr": 0.012713669449538135
},
"community|arabic_mmlu:Economics (High School)|0": {
"acc_norm": 0.7444444444444445,
"acc_norm_stderr": 0.023020332152774988
},
"community|arabic_mmlu:Economics (Middle School)|0": {
"acc_norm": 0.8160919540229885,
"acc_norm_stderr": 0.04177540678018986
},
"community|arabic_mmlu:Economics (University)|0": {
"acc_norm": 0.635036496350365,
"acc_norm_stderr": 0.041281418039994466
},
"community|arabic_mmlu:General Knowledge|0": {
"acc_norm": 0.6898148148148148,
"acc_norm_stderr": 0.015746041312239174
},
"community|arabic_mmlu:General Knowledge (Middle School)|0": {
"acc_norm": 0.7674418604651163,
"acc_norm_stderr": 0.03230654083203451
},
"community|arabic_mmlu:General Knowledge (Primary School)|0": {
"acc_norm": 0.7530864197530864,
"acc_norm_stderr": 0.03398458402215315
},
"community|arabic_mmlu:Geography (High School)|0": {
"acc_norm": 0.6059730250481695,
"acc_norm_stderr": 0.015174005775184334
},
"community|arabic_mmlu:Geography (Middle School)|0": {
"acc_norm": 0.7426470588235294,
"acc_norm_stderr": 0.026556519470041517
},
"community|arabic_mmlu:Geography (Primary School)|0": {
"acc_norm": 0.7543859649122807,
"acc_norm_stderr": 0.05752139056405265
},
"community|arabic_mmlu:History (High School)|0": {
"acc_norm": 0.4776315789473684,
"acc_norm_stderr": 0.018130679701241187
},
"community|arabic_mmlu:History (Middle School)|0": {
"acc_norm": 0.6650246305418719,
"acc_norm_stderr": 0.033208527423483104
},
"community|arabic_mmlu:History (Primary School)|0": {
"acc_norm": 0.6862745098039216,
"acc_norm_stderr": 0.04617034827006719
},
"community|arabic_mmlu:Islamic Studies|0": {
"acc_norm": 0.568075117370892,
"acc_norm_stderr": 0.019610860755030473
},
"community|arabic_mmlu:Islamic Studies (High School)|0": {
"acc_norm": 0.7335329341317365,
"acc_norm_stderr": 0.024227547959039863
},
"community|arabic_mmlu:Islamic Studies (Middle School)|0": {
"acc_norm": 0.6974789915966386,
"acc_norm_stderr": 0.02983796238829194
},
"community|arabic_mmlu:Islamic Studies (Primary School)|0": {
"acc_norm": 0.8068068068068068,
"acc_norm_stderr": 0.012497277591195375
},
"community|arabic_mmlu:Law (Professional)|0": {
"acc_norm": 0.767515923566879,
"acc_norm_stderr": 0.023876360884096257
},
"community|arabic_mmlu:Management (University)|0": {
"acc_norm": 0.8266666666666667,
"acc_norm_stderr": 0.04400382183783964
},
"community|arabic_mmlu:Math (Primary School)|0": {
"acc_norm": 0.78239608801956,
"acc_norm_stderr": 0.020427565086683957
},
"community|arabic_mmlu:Natural Science (Middle School)|0": {
"acc_norm": 0.8347107438016529,
"acc_norm_stderr": 0.023926644290131693
},
"community|arabic_mmlu:Natural Science (Primary School)|0": {
"acc_norm": 0.8869047619047619,
"acc_norm_stderr": 0.017303671012065352
},
"community|arabic_mmlu:Philosophy (High School)|0": {
"acc_norm": 0.6923076923076923,
"acc_norm_stderr": 0.07487142513727499
},
"community|arabic_mmlu:Physics (High School)|0": {
"acc_norm": 0.6039215686274509,
"acc_norm_stderr": 0.03068767658826368
},
"community|arabic_mmlu:Political Science (University)|0": {
"acc_norm": 0.6904761904761905,
"acc_norm_stderr": 0.03197777494209474
},
"community|arabic_mmlu:Social Science (Middle School)|0": {
"acc_norm": 0.5477178423236515,
"acc_norm_stderr": 0.03212754578265449
},
"community|arabic_mmlu:Social Science (Primary School)|0": {
"acc_norm": 0.7843971631205674,
"acc_norm_stderr": 0.015499186819256007
},
"community|arabic_mmlu_ht:abstract_algebra|0": {
"acc_norm": 0.38,
"acc_norm_stderr": 0.048783173121456316
},
"community|arabic_mmlu_ht:anatomy|0": {
"acc_norm": 0.5185185185185185,
"acc_norm_stderr": 0.043163785995113245
},
"community|arabic_mmlu_ht:astronomy|0": {
"acc_norm": 0.7894736842105263,
"acc_norm_stderr": 0.03317672787533157
},
"community|arabic_mmlu_ht:business_ethics|0": {
"acc_norm": 0.68,
"acc_norm_stderr": 0.04688261722621504
},
"community|arabic_mmlu_ht:clinical_knowledge|0": {
"acc_norm": 0.660377358490566,
"acc_norm_stderr": 0.02914690474779832
},
"community|arabic_mmlu_ht:college_biology|0": {
"acc_norm": 0.6875,
"acc_norm_stderr": 0.038760854559127644
},
"community|arabic_mmlu_ht:college_chemistry|0": {
"acc_norm": 0.44,
"acc_norm_stderr": 0.049888765156985884
},
"community|arabic_mmlu_ht:college_computer_science|0": {
"acc_norm": 0.63,
"acc_norm_stderr": 0.04852365870939099
},
"community|arabic_mmlu_ht:college_mathematics|0": {
"acc_norm": 0.48,
"acc_norm_stderr": 0.050211673156867795
},
"community|arabic_mmlu_ht:college_medicine|0": {
"acc_norm": 0.6011560693641619,
"acc_norm_stderr": 0.0373362665538351
},
"community|arabic_mmlu_ht:college_physics|0": {
"acc_norm": 0.5098039215686274,
"acc_norm_stderr": 0.04974229460422817
},
"community|arabic_mmlu_ht:computer_security|0": {
"acc_norm": 0.73,
"acc_norm_stderr": 0.044619604333847394
},
"community|arabic_mmlu_ht:conceptual_physics|0": {
"acc_norm": 0.7106382978723405,
"acc_norm_stderr": 0.029644006577009618
},
"community|arabic_mmlu_ht:econometrics|0": {
"acc_norm": 0.47368421052631576,
"acc_norm_stderr": 0.046970851366478626
},
"community|arabic_mmlu_ht:electrical_engineering|0": {
"acc_norm": 0.5862068965517241,
"acc_norm_stderr": 0.04104269211806232
},
"community|arabic_mmlu_ht:elementary_mathematics|0": {
"acc_norm": 0.6746031746031746,
"acc_norm_stderr": 0.024130158299762613
},
"community|arabic_mmlu_ht:formal_logic|0": {
"acc_norm": 0.5714285714285714,
"acc_norm_stderr": 0.0442626668137991
},
"community|arabic_mmlu_ht:global_facts|0": {
"acc_norm": 0.42,
"acc_norm_stderr": 0.049604496374885836
},
"community|arabic_mmlu_ht:high_school_biology|0": {
"acc_norm": 0.7175452399685287,
"acc_norm_stderr": 0.007291596820938285
},
"community|arabic_mmlu_ht:high_school_chemistry|0": {
"acc_norm": 0.7158864541832669,
"acc_norm_stderr": 0.007117463813276186
},
"community|arabic_mmlu_ht:high_school_computer_science|0": {
"acc_norm": 0.8,
"acc_norm_stderr": 0.04020151261036846
},
"community|arabic_mmlu_ht:high_school_european_history|0": {
"acc_norm": 0.5976447497546614,
"acc_norm_stderr": 0.005431514510405362
},
"community|arabic_mmlu_ht:high_school_geography|0": {
"acc_norm": 0.803030303030303,
"acc_norm_stderr": 0.028335609732463362
},
"community|arabic_mmlu_ht:high_school_government_and_politics|0": {
"acc_norm": 0.8082901554404145,
"acc_norm_stderr": 0.028408953626245282
},
"community|arabic_mmlu_ht:high_school_macroeconomics|0": {
"acc_norm": 0.7295053614666206,
"acc_norm_stderr": 0.008263131293548422
},
"community|arabic_mmlu_ht:high_school_mathematics|0": {
"acc_norm": 0.42962962962962964,
"acc_norm_stderr": 0.030182099804387262
},
"community|arabic_mmlu_ht:high_school_microeconomics|0": {
"acc_norm": 0.7184873949579832,
"acc_norm_stderr": 0.02921354941437216
},
"community|arabic_mmlu_ht:high_school_physics|0": {
"acc_norm": 0.5033112582781457,
"acc_norm_stderr": 0.04082393379449654
},
"community|arabic_mmlu_ht:high_school_psychology|0": {
"acc_norm": 0.7651376146788991,
"acc_norm_stderr": 0.018175110510343578
},
"community|arabic_mmlu_ht:high_school_statistics|0": {
"acc_norm": 0.7164461247637051,
"acc_norm_stderr": 0.0069292790274951555
},
"community|arabic_mmlu_ht:high_school_us_history|0": {
"acc_norm": 0.7745098039215687,
"acc_norm_stderr": 0.029331162294251742
},
"community|arabic_mmlu_ht:high_school_world_history|0": {
"acc_norm": 0.8396624472573839,
"acc_norm_stderr": 0.02388438092596567
},
"community|arabic_mmlu_ht:human_aging|0": {
"acc_norm": 0.672645739910314,
"acc_norm_stderr": 0.03149384670994131
},
"community|arabic_mmlu_ht:human_sexuality|0": {
"acc_norm": 0.6335877862595419,
"acc_norm_stderr": 0.04225875451969637
},
"community|arabic_mmlu_ht:international_law|0": {
"acc_norm": 0.8016528925619835,
"acc_norm_stderr": 0.036401182719909476
},
"community|arabic_mmlu_ht:jurisprudence|0": {
"acc_norm": 0.7314814814814815,
"acc_norm_stderr": 0.042844679680521934
},
"community|arabic_mmlu_ht:logical_fallacies|0": {
"acc_norm": 0.6871165644171779,
"acc_norm_stderr": 0.03642914578292406
},
"community|arabic_mmlu_ht:machine_learning|0": {
"acc_norm": 0.48214285714285715,
"acc_norm_stderr": 0.047427623612430116
},
"community|arabic_mmlu_ht:management|0": {
"acc_norm": 0.6796116504854369,
"acc_norm_stderr": 0.04620284082280041
},
"community|arabic_mmlu_ht:marketing|0": {
"acc_norm": 0.8034188034188035,
"acc_norm_stderr": 0.02603538609895129
},
"community|arabic_mmlu_ht:medical_genetics|0": {
"acc_norm": 0.63,
"acc_norm_stderr": 0.04852365870939099
},
"community|arabic_mmlu_ht:miscellaneous|0": {
"acc_norm": 0.743801652892562,
"acc_norm_stderr": 0.008875621982917577
},
"community|arabic_mmlu_ht:moral_disputes|0": {
"acc_norm": 0.6820809248554913,
"acc_norm_stderr": 0.025070713719153186
},
"community|arabic_mmlu_ht:moral_scenarios|0": {
"acc_norm": 0.506145251396648,
"acc_norm_stderr": 0.01672123848363141
},
"community|arabic_mmlu_ht:nutrition|0": {
"acc_norm": 0.7222222222222222,
"acc_norm_stderr": 0.025646863097137904
},
"community|arabic_mmlu_ht:philosophy|0": {
"acc_norm": 0.707395498392283,
"acc_norm_stderr": 0.025839898334877976
},
"community|arabic_mmlu_ht:prehistory|0": {
"acc_norm": 0.6728395061728395,
"acc_norm_stderr": 0.02610567386140983
},
"community|arabic_mmlu_ht:professional_accounting|0": {
"acc_norm": 0.7002658396101019,
"acc_norm_stderr": 0.006819725978102662
},
"community|arabic_mmlu_ht:professional_law|0": {
"acc_norm": 0.5940903968949544,
"acc_norm_stderr": 0.005495108555167751
},
"community|arabic_mmlu_ht:professional_medicine|0": {
"acc_norm": 0.7354917532070862,
"acc_norm_stderr": 0.010904778852241306
},
"community|arabic_mmlu_ht:professional_psychology|0": {
"acc_norm": 0.7116757065372538,
"acc_norm_stderr": 0.0076546196855586295
},
"community|arabic_mmlu_ht:public_relations|0": {
"acc_norm": 0.5818181818181818,
"acc_norm_stderr": 0.04724577405731573
},
"community|arabic_mmlu_ht:security_studies|0": {
"acc_norm": 0.7142857142857143,
"acc_norm_stderr": 0.028920583220675596
},
"community|arabic_mmlu_ht:sociology|0": {
"acc_norm": 0.746268656716418,
"acc_norm_stderr": 0.03076944496729602
},
"community|arabic_mmlu_ht:us_foreign_policy|0": {
"acc_norm": 0.82,
"acc_norm_stderr": 0.038612291966536934
},
"community|arabic_mmlu_ht:virology|0": {
"acc_norm": 0.5301204819277109,
"acc_norm_stderr": 0.03885425420866767
},
"community|arabic_mmlu_ht:world_religions|0": {
"acc_norm": 0.7134502923976608,
"acc_norm_stderr": 0.03467826685703826
},
"community|aratrust:Ethics|0": {
"acc_norm": 0.8166666666666667,
"acc_norm_stderr": 0.05037523978531017
},
"community|aratrust:Illegal|0": {
"acc_norm": 0.9056603773584906,
"acc_norm_stderr": 0.040534804190911294
},
"community|aratrust:MentalHealth|0": {
"acc_norm": 0.9210526315789473,
"acc_norm_stderr": 0.031137262016313737
},
"community|aratrust:Offensive|0": {
"acc_norm": 0.855072463768116,
"acc_norm_stderr": 0.042689636687312106
},
"community|aratrust:PhysicalHealth|0": {
"acc_norm": 0.9178082191780822,
"acc_norm_stderr": 0.03236860017981242
},
"community|aratrust:Privacy|0": {
"acc_norm": 0.9649122807017544,
"acc_norm_stderr": 0.024588241692166144
},
"community|aratrust:Trustfulness|0": {
"acc_norm": 0.782051282051282,
"acc_norm_stderr": 0.047048893371093484
},
"community|aratrust:Unfairness|0": {
"acc_norm": 0.9090909090909091,
"acc_norm_stderr": 0.03912104390108503
},
"community|madinah_qa:Arabic Language (General)|0": {
"acc_norm": 0.7941176470588235,
"acc_norm_stderr": 0.01635804429747852
},
"community|madinah_qa:Arabic Language (Grammar)|0": {
"acc_norm": 0.31232876712328766,
"acc_norm_stderr": 0.024291018800569122
},
"community|alghafa:_average|0": {
"acc_norm": 0.7798376497688053,
"acc_norm_stderr": 0.013622435279038166
},
"community|arabic_mmlu:_average|0": {
"acc_norm": 0.6957761961875122,
"acc_norm_stderr": 0.0324229297443301
},
"community|arabic_mmlu_ht:_average|0": {
"acc_norm": 0.6572997736047431,
"acc_norm_stderr": 0.03149671003952715
},
"community|aratrust:_average|0": {
"acc_norm": 0.884039353799281,
"acc_norm_stderr": 0.03848296522800054
},
"community|madinah_qa:_average|0": {
"acc_norm": 0.5532232070910555,
"acc_norm_stderr": 0.02032453154902382
}
}
Dataset Details
Dataset Description
- Curated by: [More Information Needed]
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
Dataset Sources [optional]
- Repository: [More Information Needed]
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
Uses
Direct Use
[More Information Needed]
Out-of-Scope Use
[More Information Needed]
Dataset Structure
[More Information Needed]
Dataset Creation
Curation Rationale
[More Information Needed]
Source Data
Data Collection and Processing
[More Information Needed]
Who are the source data producers?
[More Information Needed]
Annotations [optional]
Annotation process
[More Information Needed]
Who are the annotators?
[More Information Needed]
Personal and Sensitive Information
[More Information Needed]
Bias, Risks, and Limitations
[More Information Needed]
Recommendations
Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations.
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Dataset Card Authors [optional]
[More Information Needed]
Dataset Card Contact
[More Information Needed]