language:

• en
  task_categories:
• question-answering
  tags:
• arifOS
• governance
• benchmark
• industrial-AI
• constitutional-AI
• asset-operations
• AssetOpsBench
  license: apache-2.0
  pretty_name: a2b-eval-results

configs:

• config_name: per_scenario
  data_files:

  • split: train
    path: data/per_scenario/results.jsonl
    splits:
  • name: train
    num_examples: 102

• config_name: run_summary
  data_files:

  • split: train
    path: data/run_summary/runs.jsonl
    splits:
  • name: train
    num_examples: 3

• config_name: default
  data_files:

  • split: train
    path: evals/azure-gpt4.1-mini-eval-20260715T072440Z.jsonl
    splits:
  • name: train
    num_examples: 111

• config_name: scenario_data
  data_files:

  • split: train
    path: scenario_data/sample_50_questions.jsonl
    splits:
  • name: train
    num_examples: 50

• config_name: cutoff_m3_three_lane_2026_10_05
  data_files:

  • split: train
    path: evals/cutoff-m3-three-lane-20261005T135500Z.jsonl
    splits:
  • name: train
    num_examples: 60

50

🗺️ Position in I-ARIF Governance Stack

This dataset is part of the arifOS constitutional governance training-and-evaluation pipeline — a closed-loop alignment substrate.

Total: 1,122 downloads across 8 datasets · verified 2026-08-11
🔗 Full stack: https://huggingface.co/ariffazil · 🌐 Federation: https://arif-fazil.com

a2b-eval-results

Structured eval results from the arifOS × AssetOpsBench bridge evaluation.

Source

Generated by ariffazil/a2b harness (eval_harness.py).

• arifOS kernel: v2026.05.05-SSCT
• Model: MiniMax-M3 via TokenRouter
• Benchmark: AssetOpsBench FailureSensorIQ (50 MCQ scenarios)
• Runs: 3 (governance ON, governance OFF, smoke test)

Results Summary

Key Findings (OBS Only)

• Governance held every scenario: 50/50 HOLD ✅
• Zero seals written: T1 identity not registered (airlock working as designed)
• A-bias confirmed: model defaults to "A" at 38–42% (expected 25%)
• Same accuracy: governance does not degrade LLM performance (32% vs 36% = noise)

Dataset Structure

Two configs (subsets):

per_scenario — Individual scenario results (102 rows)

Load with:

from datasets import load_dataset
ds = load_dataset("ariffazil/a2b-eval-results", "per_scenario")

run_summary — Run-level aggregate metrics (3 rows)

Load with:

from datasets import load_dataset
ds = load_dataset("ariffazil/a2b-eval-results", "run_summary")

Scenario Data

scenario_data/sample_50_questions.jsonl — the 50 FailureSensorIQ MCQ scenarios from AssetOpsBench.

Usage

from datasets import load_dataset

# Per-scenario results
ds = load_dataset("ariffazil/a2b-eval-results", "per_scenario")
print(f"Total scenarios: {len(ds['train'])}")

# Filter governance ON
gov = ds["train"].filter(lambda x: x["governance_enabled"])
print(f"Governance scenarios: {len(gov)}")

# Accuracy by run
for run_id in ["run001_gov", "run002_nogov"]:
    run_rows = ds["train"].filter(lambda x: x["run_id"] == run_id)
    correct = sum(1 for r in run_rows if r["is_correct"])
    print(f"{run_id}: {correct}/{len(run_rows)} = {correct/len(run_rows):.2%}")

# Run summaries
summaries = load_dataset("ariffazil/a2b-eval-results", "run_summary")
for row in summaries["train"]:
    print(f"{row['run_id']}: accuracy={row['accuracy']}, holds={row['n_governance_holds']}")

Related

• a2b GitHub repo — harness code + docs
• arifOS Federation — constitutional kernel
• AssetOpsBench (IBM) — upstream benchmark

DITEMPA BUKAN DIBERI — Forged, Not Given.

2026-10-05 — arifOS evidence batch (cutoff test + 4 discovery audits)

F13-directed evidence package, 333-AGI session SEAL-121c33c37b4a4436:

One-line verdict (from the PDF):

A frozen model cannot know the present (H1 held, 0–17% on post-cutoff facts across all lanes), but governance changes how it fails: the arifOS membrane delivered the best calibration (Brier 0.2057 vs bare 0.2117 vs harness 0.2531), 0% false-rejections of reality, 0% gullibility, 0% fabrication, and 98.6% classified-UNKNOWN on post-cutoff — while the external ReAct-class harness made things worse than nothing (Brier +0.041 over bare, FAIL — blocked from constitutional lanes).

Convention source: /root/AAA/instructions/correction-ledger-convention.md (federation box).