license: mit
language: da
tags:

• danish
• grpo
• instruction-following
• if-eval
  task_categories:
• text-generation
  size_categories:
• 1K<n<100K

danish-if-grpo-combined-v3

10,000 Danish instruction-following prompts for GRPO reward training,
built by rewriting danish-instruction-following-v4 prompts with a
sampled mix of our 46 Danish constraints + ~24 Google IFEval-schema
constraints.

What's new vs v2

Structural fixes (see scripts/constraint_compat.py on
github):

• 0% impossible constraint combos (v2 had 15.2%; e.g. constrained_response
  paired with 20+ content-shape rules that can't co-satisfy). Enforced via a
  reject-sampling loop with an explicit alias-map + pair-incompatibility table.
• Namespace deduped — no more rows with both google:startend:quotation
  and its custom alias entire_in_quotes (v2 had 260+).

Language / quality fixes (Gemini template + verify-retry):

• English contamination on case rules 95% → 0.1% — english_capital /
  english_lowercase descriptions swapped to Danish (verifier is already
  Danish-aware; only the source text was steering the model to English).
• English constraint verbatim 11.3% → 4.5% — explicit "OVERSÆT engelske
  regler til dansk" instruction to the rewriter.
• Missing task text 9.3% → 6.5% — explicit "GRUNDOPGAVEN skal fremgå
  TYDELIGT" requirement.
• Constraint dropped from prompt 17.6% → 12.8% — verify-retry loop:
  after Gemini returns, keyword-scan the new prompt against each constraint;
  if any missing, retry once with "you forgot X" hint.

Recipe

• Source: jensjepsen/danish-instruction-following-v4 (extract raw task,
  discard old rules)
• Constraint pool: our-46 Danish + ~24 Google IFEval; 50/50 mix, n_rules
  sampled 1-5 uniform
• Rewriter: google/gemini-2.5-flash-lite via OpenRouter (verified
  6.25× cheaper on output than 3.5-flash-lite and also better at not
  dropping constraints)
• Style variation: 8 orthogonal axes (placement, form, opener,
  terminator, case, quirk, tone, rephrase) cartesian-sampled per row —
  same as v2

Schema

task           : str  — the base task (post-extraction)
prompt         : str  — the new full Danish prompt (task + rules woven)
constraints    : list[str]  — canonical constraint names
params         : list[dict] — per-constraint kwargs
n_rules        : int
style_tags     : str (JSON) — 8-axis style tags used to steer the rewrite
source         : str — v4 task type (e.g. wiki:brainstorming, sciq)
old_constraints: list[str] — the rules from the source v4 row (discarded, kept for provenance)
row_idx        : int

Constraint distribution (top 15)

Same shape as v2. Top offenders that used to break v2 (constrained_response,
json_format paired with wrappers, two_responses × quotation) are now
rejected at sample time.

Known residuals

• ~12.8% of rows still have Gemini paraphrasing a constraint too subtly for
  strict keyword matching. Manual sampling suggests most are true paraphrases
  ("nævne" instead of "inkludér", "skal indgå" instead of "brug ordet") —
  not real drops. Real drop rate probably closer to 5-8%.
• 4.5% still contain English constraint text verbatim (some Google
  descriptions Gemini won't Danish-ify even under prompting).

Related

• v2 (previous): jensjepsen/danish-if-grpo-combined-v2
• JSON GRPO reward source: jensjepsen/danish-json-grpo-v1
• SFT IF source: jensjepsen/danish-instruction-following-v4