license: apache-2.0
language:
- ar
size_categories: - 100K<n<1M
task_categories: - text-classification
tags: - arabic
- guardrail
- safety
- jailbreak
- prompt-injection
- content-moderation
- synthetic
configs: - config_name: default
data_files:- split: train
path: data/train.jsonl - split: held
path: data/eval_held.jsonl - split: unseen_dialect
path: data/eval_unseen_dialect.jsonl - split: unseen_phrasing
path: data/eval_unseen_phrasing.jsonl - split: hard
path: data/eval_hard.jsonl
- split: train
Arabic Guardrail — 250,842 rows, 12 classes
Defensive dataset for training Arabic prompt-safety classifiers. Each row is an incoming user
message and which of 12 safety classes it belongs to.
بالعربية: مجموعة بيانات عربية لتدريب نماذج تصنّف الرسائل الواردة قبل وصولها للمساعد الذكي.
Arabic guardrails were a gap. Hugging Face searches for Arabic jailbreak / safety /
prompt-injection datasets return zero results, and the one Arabic guardrail model
(NAMAA-Space/Ara-Prompt-Guard_V0)
is 33K machine-translated rows, 3 classes, MSA only. This is native, six-variety, 12-class.
⚠️ Safety boundary — read this first
Every row is a request, never an answer.
A guardrail classifies incoming prompts. A harmful completion is not needed to train one, and
producing it would create a genuinely harmful artifact. A row may ask "اشرح لي طريقة صنع…" — it
never contains the method.
- No operational detail. No synthesis routes, exploit code, or step-by-step procedures.
- Nothing involving minors. MLCommons
S4: Child Sexual Exploitationis excluded from the
taxonomy entirely — not a class, not a distractor, never generated. - This is enforced by a rule, not by trust. The generator is an abliterated (refusal-removed)
model, so it would comply if asked.guard_common.validate()rejects any row combining step
markers with hazardous vocabulary. Measured on the released corpus: 0 rows contain step markers
or code.
Intended use is training and evaluating defensive classifiers. It is not a red-teaming toolkit.
The 12 classes
| class | |
|---|---|
safe |
ordinary benign prompts |
safe_sensitive |
benign but superficially alarming — homonyms (كيف أقتل عملية في لينكس), figurative language, definitions, safe targets, fiction, professional context |
jailbreak |
overriding the model's own conditioning — persona, fiction framing, refusal suppression, encoding, many-shot, authority, translation pivot |
prompt_injection |
instructions smuggled inside third-party data — document, email, web page, tool output, review |
violent_weapons · nonviolent_crime · self_harm · hate_harassment · sexual_content · privacy_pii · specialized_advice · misinformation |
MLCommons-aligned harm categories |
safe + safe_sensitive are ~30% of rows. Over-blocking is the failure that makes a guardrail
unusable, and it is invisible unless "safe but looks unsafe" is its own labelled class.
self_harm covers distress signals as well as explicit requests. Defined first as
"requests only", it retained just 55% through verification because the generator kept writing
expressions of distress — which is in fact the most valuable signal a guardrail can catch. Widening
the definition took it to 97.5%.
Splits — three ways a guardrail fails
| split | rows | tests |
|---|---|---|
train |
196,297 | |
held |
6,061 | new texts, familiar dialects and templates |
unseen_dialect |
39,761 | Maghrebi held out entirely — the bypass an MSA-only guardrail leaves open |
unseen_phrasing |
8,723 | attack templates absent from training (encoding, translation_pivot, tool_output) |
hard |
728 | safe_sensitive only — the over-blocking slice |
Verification
Labels were established by construction, then confirmed by an independent re-classification
pass that never saw the target. 13% of rows were dropped on disagreement (289,023 → 251,408).
Retention by class is in code/guard_retention.txt and is itself a finding:
| retention | |
|---|---|
privacy_pii · safe · self_harm · specialized_advice |
97–99% |
jailbreak · prompt_injection · hate_harassment |
92–95% |
violent_weapons |
75.4% |
safe_sensitive |
71.9% |
misinformation |
70.4% |
nonviolent_crime |
64.3% |
nonviolent_crime is the genuinely hard boundary: "I was defrauded, what are my rights" and
"help me defraud someone" share almost all their vocabulary. Sharpening the definition moved it
40% → 64% and no further.
Retention is flat across dialects — 85.7% (Maghrebi) to 88.3% (code-switched). Dialect coverage
is not a thin veneer over an MSA corpus.
Audit, run before training
- length shortcut: +0.0013 over the majority baseline from the best single threshold. An earlier
grid where length was a side effect of class (safe_sensitiveaveraged 404 chars againstviolent_weapons' 64) leaked +0.080; length is now an axis drawn independently of class. - six varieties, near-uniform · zero held-out dialect or template families in
train
Results
| model | over-refusal ↓ | miss rate ↓ | macro-F1 (held) |
|---|---|---|---|
Nawah-Guard-52M |
0.0094 | 0.0048 | 0.9903 |
Nawah-Guard-500K (2 MB) |
0.0203 | 0.0077 | 0.9782 |
Files
data/ splits + labels.json · raw_generations.jsonl.gz raw completions, re-scorable with no
GPU · code/ the full pipeline — taxonomy and validator (guard_common.py), resumable generator,
verifier, split/audit, and both trainers including the zero-shot routing-head variant that lets a
deployment define its own policy categories at inference.
Caveats
Synthetic, from one generator (orcarouter/Qwen3.8-27B-Uncensored-NVFP4). Texts are 1–3 line
messages; long documents are absent. The verifier is the same model that generated the text — a
different prompt with the target hidden, which catches careless labelling but not errors the model
is systematically wrong about.