DarijaLLM Evaluation Benchmark
Dataset Summary
This dataset contains the evaluation results of multiple Large Language Models (Gemma 3 1B, Gemma 4 2B, and Gemma 4 4B) fine-tuned on the Northern Moroccan (Chamali) dialect.
The evaluation was conducted using an "LLM-as-a-Judge" pipeline powered by Gemini 3 Flash.
Data Structure
The JSON file contains 100 benchmark questions divided into categories (Science, Culture, Jailbreaks, etc.), alongside the answers generated by each model, and the final scores (1-10) assigned by the judge based on:
- Dialect Purity
- Task Success
- Natural Flow
How to use it in Python
You can easily load this benchmark using the Hugging Face datasets library:
from datasets import load_dataset
# Replace 'younesAbaaoud' with your actual Hugging Face username
dataset = load_dataset("younesAbaaoud/DarijaLLM-Eval-Benchmark")
print(dataset)