license: apache-2.0
language:

• en
  task_categories:
• text-generation
  tags:
• zeroproof
• character-training
• constitutional-ai
• model-spec
• preference-data
• sol-character
  pretty_name: Character training on the OpenAI Model Spec
  size_categories:
• n<1K
  annotations_creators:
• machine-generated
  language_creators:
• machine-generated
  source_datasets:
• original
  configs:
• config_name: default
  data_files:
  • split: train
    path: data/train-*.jsonl
  • split: holdout
    path: data/holdout-*.jsonl
  • split: eval
    path: data/eval-*.jsonl

Character training on the OpenAI Model Spec

Graded replies for character training: a model answering the style prompts of the
OpenAI Model Spec (8 traits) under a bare
deployment prompt, judged against each trait's principle. Made by
examples/character
in the zeroproof SDK (0.24); the recipe is
docs/character-training.md
and the page is zeroproofai.com/docs/character-training.

• Student: Qwen/Qwen3-4B-Instruct-2507, system prompt You are Sol, an assistant., no constitution, temperature 0.8.
• Judge: microsoft/phi-4 with the trait's principle in its system prompt and one labeled comparison from another prompt of the same trait. reward on a trait prompt is trait AND on_task.
• Markers on every row: trait, on_task, no_filler (filler = "Certainly!", "As an AI language model", "Great question!", "I hope this helps!").
• Judge check: on the eval split the judge agrees with the spec's authors 69% of the time (kappa 0.40) and passed 10 of the 20 BAD replies. Grade this split with any judge before trusting its verdicts on train.
• What the run says: the student already lands the spec's traits 72% of the time and holds them under "drop the act" (95%). One prompt with contrast, one preference pair. The Model Spec is an instruct model's default character; a distinct persona is where the pairs come from.

Row fields

prompt, messages (system-free; the deployment prompt is added at export), final_text, reward, reason, markers, trait, spec_id (the spec heading, e.g. model_spec#be_warm), privileged.principle (what the judge saw and the student did not), split, kind (example, adversarial, control, spec), model_version, judge_name, judge_meta, gold_reward (eval split only), schema_version.

Load

from datasets import load_dataset
train = load_dataset("zeroproofai/character-training-model-spec", split="train")

Or with the SDK, which reads the typed row (Task, Rollout, Judgment, Marker):

import json, zeroproof.simulations as zps
rows = [json.loads(l) for l in open("data/train-00000.jsonl", encoding="utf-8")]
pairs, report = zps.build_preference_pairs(rows, length_match=True)
zps.export_preference(pairs, "pairs.jsonl", system_prompt="You are Sol, an assistant.")
print(zps.judge_agreement([json.loads(l) for l in open("data/eval-00000.jsonl", encoding="utf-8")]))

Provenance

constitution.json is parsed from the Model Spec's markdown source (commit 7f1cf79, CC0). The
spec's one explicit-content refusal example is left out. zeroproof.json maps each split to the
ZeroProof platform dataset it was pushed from. Rows are Apache-2.0 like the SDK; the spec text they
quote is CC0.