extract 0 0 0

memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25

published by @memo-ozdincer

Alignment dataset tracked in the /datasets sub-catalog.

Lifetime downloads
9.6K
Last 30 days
2.1K
Likes
-
Size
3.2 GB
Created on HF
2026-05-26
Age
4 months ago

Description

JEPA-Align: Qwen3-32B Safety Defense Matrix

The complete 11-condition Qwen3-32B experiment for Predictive Representation

Alignment (PRA), the paired-view objective introduced in Predictive

Representation Alignment Improves Generalization in LLM Safety.

PRA aligns adversarially rewritten prompts with clean prompts expressing the

same intent. This release contains trained adapters, attack traces, benign

capability evaluations, machine-readable results, and paper-ready tables for… See the full description on the dataset page: https://huggingface.co/datasets/memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25.

Tags

task_categories:text-classificationlanguage:enlicense:cc-by-4.0size_categories:n<1Kformat:csvmodality:tabularmodality:textlibrary:datasetslibrary:pandaslibrary:polarslibrary:mlcroissantregion:usjailbreakadversarial-robustnessllm-safetyrepresentation-learningjepapraqwen3

README current version from Hugging Face


license: cc-by-4.0
language:

  • en
    tags:
  • jailbreak
  • adversarial-robustness
  • llm-safety
  • representation-learning
  • jepa
  • pra
  • qwen3
    task_categories:
  • text-classification
    pretty_name: "JEPA-Align: Qwen3-32B Safety Defense Matrix"
    size_categories:
  • n<1K
    configs:
  • config_name: default
    data_files:
    • split: train
      path: consolidated_results.csv

JEPA-Align: Qwen3-32B Safety Defense Matrix

The complete 11-condition Qwen3-32B experiment for Predictive Representation
Alignment (PRA)
, the paired-view objective introduced in Predictive
Representation Alignment Improves Generalization in LLM Safety
.

PRA aligns adversarially rewritten prompts with clean prompts expressing the
same intent. This release contains trained adapters, attack traces, benign
capability evaluations, machine-readable results, and paper-ready tables for
the full 32B defense matrix.

Results at a glance

  • 18.1% → 0.7% mean attack success from the undefended model to the best
    PRA-enhanced defenses.
  • PRA reduces CE-floor from 1.5% to 0.7% mean ASR.
  • PRA reduces Circuit Breakers from 2.9% to 0.7% mean ASR with the identity
    predictor.
  • The strongest PRA cells preserve Qwen3-32B performance across MMLU, GSM8K,
    TruthfulQA, and HellaSwag.
  • The matrix evaluates 11 conditions across three attack settings and four
    benign-capability benchmarks.

Full defense matrix

Attack success rate (ASR) is evaluated with StrongREJECT under strict
dual-context validation. Lower is better. Benign scores are reported as
fractions; higher is better.

Condition Defense PRA Pair data Direct Prefill Template jailbreak Mean ASR MMLU GSM8K TQA-MC2 HellaSwag
q_base None — — 8.3 36.5 9.4 18.1 .833 .740 .570 .700
q_ce CE-floor No WJ 1.0 3.0 0.6 1.5 .834 .765 .575 .695
q_cb Circuit Breakers No WJ 3.0 4.0 1.7 2.9 .832 .765 .571 .690
q_ce_pra CE-floor MLP WJ 1.0 1.0 0.2 0.7 .836 .735 .574 .695
q_ce_pra_id CE-floor Identity WJ 1.0 1.0 0.2 0.7 .827 .710 .593 .675
q_cb_pra Circuit Breakers MLP WJ 4.0 3.0 1.8 2.9 .831 .855 .566 .695
q_cb_pra_id Circuit Breakers Identity WJ 1.0 1.0 0.1 0.7 .830 .820 .568 .700
q_ce_pra_jepa CE-floor MLP JepaData 1.0 3.0 0.4 1.5 .836 .775 .573 .695
q_ce_pra_id_jepa CE-floor Identity JepaData 1.0 1.0 0.1 0.7 .810 .330 .606 .695
q_cb_pra_jepa Circuit Breakers MLP JepaData 3.0 3.0 0.8 2.2 .833 .865 .546 .685
q_cb_pra_id_jepa Circuit Breakers Identity JepaData 1.0 3.0 0.4 1.5 .821 .850 .579 .705

The identity-predictor CE-floor/JepaData condition is the sole capability
outlier on GSM8K. The other defended conditions preserve the four evaluated
capability benchmarks near the base model. Full per-condition discussion is in
RESULTS.md.

Experimental design

  • Base model: Qwen3-32B
  • Defense objectives: CE-floor and Circuit Breakers
  • PRA predictors: two-layer MLP and identity mapping
  • Paired views: WildJailbreak and JepaData
  • Attacks: direct, prefilling, and 20-template PyRIT jailbreaks on 100
    AdvBench behaviors
  • Judge: StrongREJECT with strict dual-context validation
  • Capability: MMLU, GSM8K, TruthfulQA-MC2, and HellaSwag
  • Training: LoRA, matched data and optimization settings within each
    PRA/no-PRA comparison
  • Compute: eight NVIDIA H200 GPUs

What is included

consolidated_results.csv/json     Complete machine-readable matrix
paper_snippets/                   LaTeX tables and result macros
q_base/                           Undefended Qwen3-32B evaluations
q_ce/, q_cb/                      Matched defenses without PRA
q_*_pra*/                         PRA adapters, predictors, and evaluations
wang_probe/                       Out-of-distribution representation probes

Each trained condition includes its LoRA adapter, exact hyperparameters and
data provenance, training metrics, attack traces, judge scores, and benign
evaluation outputs.

Load the result table

from datasets import load_dataset

results = load_dataset(
    "memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25",
    split="train",
)
print(results.to_pandas())

Related artifacts

Citation

@misc{ozdincer2026jepaalign32b,
  title        = {JEPA-Align: Qwen3-32B Safety Defense Matrix},
  author       = {Ozdincer, Memo},
  year         = {2026},
  howpublished = {Hugging Face Dataset},
  url          = {https://huggingface.co/datasets/memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25}
}

README history 20 revisions

Every night we snapshot the README of every dataset in the catalog. When the SHA changes we archive the new version and diff it against the last. This is the evolving thought record of the alignment-data field: what the author decided to say about the corpus, and how that framing shifted over time.

  1. 2026-08-13Polish public dataset carde37a3c35 KB
    Loading...
  2. 2026-05-26Wang strict (arXiv 2509.03888) Table 1: PRA+proj scores 91.5% HarmBench / 94....c420e349.7 KB
    Loading...
  3. 2026-05-26Wang probe v2: add MultiJail axis + 8B PRA companion (plain + proj)7500f1b9.7 KB
    Loading...
  4. 2026-05-26Add Wang et al. 'Why Probing Fails' protocol on q_base/q_ce/q_cb6a652ef9.7 KB
    Loading...
  5. 2026-05-26Remove per_cell_summary/ (replaced by consolidated_results)30593419.7 KB
    Loading...
  6. 2026-05-26Remove superseded results_summary.json828534a9.7 KB
    Loading...
  7. 2026-05-26Remove superseded submission_manifest.json2a0d9a39.7 KB
    Loading...
  8. 2026-05-26Remove superseded jobs.jsonl515084c9.7 KB
    Loading...
  9. 2026-05-26Remove superseded PROGRESS.md878e3b19.7 KB
    Loading...
  10. 2026-05-26Remove superseded INDEX.md2c17cb99.7 KB
    Loading...
  11. 2026-05-26Remove superseded EXPERIMENT_README.md254ef649.7 KB
    Loading...
  12. 2026-05-26Add cell q_cb_pra_id_jepa15f8c509.7 KB
    Loading...
  13. 2026-05-26Add cell q_cb_pra_jepac9371149.7 KB
    Loading...
  14. 2026-05-26Add cell q_ce_pra_id_jepaef55af79.7 KB
    Loading...
  15. 2026-05-26Add cell q_ce_pra_jepa640885f9.7 KB
    Loading...
  16. 2026-05-26Add cell q_cb_pra_idfd2e3f79.7 KB
    Loading...
  17. 2026-05-26Add cell q_cb_pra61fb6289.7 KB
    Loading...
  18. 2026-05-26Add cell q_ce_pra_idcad51659.7 KB
    Loading...
  19. 2026-05-26Add cell q_ce_pra42f310f9.7 KB
    Loading...
  20. 2026-05-26Add cell q_basee1a4c3b9.7 KB
    Loading...

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Community discussions 1 thread · 1 open · 1 comments

Threads posted by the community on Hugging Face: bug reports, questions about usage, pull requests on the README, license clarifications. Click any thread to expand and read the full conversation inline. Newest first.

  1. 2026-05-27[bot] Conversion to Parquetopen1 💬#1
    Loading...

Related datasets same stage · extract

Read further

Metadata

License
cc-by-4.0

Repository

Download the dataset

0

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python)
from datasets import load_dataset
ds = load_dataset("memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25")
Raw snapshot (Python)
from huggingface_hub import snapshot_download
snapshot_download(repo_id="memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25", repo_type="dataset")
git clone (requires git-lfs)
git clone https://huggingface.co/datasets/memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration