misc 0 0 0

vkatg/dag_remediation_traces

published by @vkatg

Alignment dataset tracked in the /datasets sub-catalog.

Lifetime downloads
257
Last 30 days
28
Likes
-
Size
7.6 MB
Created on HF
2026-03-10
Age
7 months ago

Description

DAG Remediation Traces

Author: Venkata Krishna Azith Teja Ganti

Part of the ExposureGuard PHI Re-identification Risk Ecosystem

Input/output execution traces for budget-constrained PHI remediation planning over multimodal clinical records. Each record pairs a patient risk profile with a complete DAG planning trace: which actions were selected, which dependency injections fired, the topological execution order, and the final residual risk and cost.

Use this to train or benchmark… See the full description on the dataset page: https://huggingface.co/datasets/vkatg/dag_remediation_traces.

Tags

task_categories:text-classificationtask_categories:otherlanguage:enlicense:mitsize_categories:1K<n<10Kformat:jsonmodality:tabularmodality:textlibrary:datasetslibrary:pandaslibrary:polarslibrary:mlcroissantregion:usphide-identificationclinical-nlpprivacydagremediation-planninggraphplanninghealthcarehipaamultimodalsyntheticbudget-constrainedtopologydependency-injection

README current version from Hugging Face


language:

  • en
    license: mit
    pretty_name: DAG Remediation Traces
    tags:
  • phi
  • de-identification
  • clinical-nlp
  • privacy
  • dag
  • remediation-planning
  • graph
  • planning
  • healthcare
  • hipaa
  • multimodal
  • synthetic
  • budget-constrained
  • topology
  • dependency-injection
    size_categories:
  • 1K<n<10K
    task_categories:
  • text-classification
  • other
    configs:
  • config_name: default
    data_files:
    • split: train
      path: data/train.jsonl
    • split: test
      path: data/test.jsonl
  • config_name: hard
    data_files:
    • split: test
      path: data/hard.jsonl

DAG Remediation Traces

Author: Venkata Krishna Azith Teja Ganti
Part of the ExposureGuard PHI Re-identification Risk Ecosystem

Input/output execution traces for budget-constrained PHI remediation planning over multimodal clinical records. Each record pairs a patient risk profile with a complete DAG planning trace: which actions were selected, which dependency injections fired, the topological execution order, and the final residual risk and cost.

Use this to train or benchmark alternative remediation planners, study cost/risk tradeoffs, or evaluate DAG construction under budget constraints across text, image, audio, and EHR modalities.

Splits

Split Config Records plan_ready below_threshold
train default 6,000 5,051 (84.2%) 949 (15.8%)
test default 1,500 1,277 (85.1%) 223 (14.9%)
test hard 1,000 1,000 (100%) 0

The hard config fixes input risk between 0.75 and 0.99 and caps budget at 0.60, creating a stress-test set where every record requires planning under tight constraints.

Plan quality

Numbers are shown for all records and for planned-only (records where the planner actually ran). The gap between the two reflects the 15.8% of default train records that were already below the risk threshold and received empty plans. The hard split has no below-threshold records so both columns are the same.

Metric default all default planned hard all hard planned
avg actions per plan 3.47 4.12 5.50 5.50
avg topological depth 3.47 4.12 5.50 5.50
max topological depth 10 10 10 10
avg residual risk 0.0420 0.0261 0.0429 0.0429
avg total cost 0.3093 0.3093 0.4678 0.4678
plans reaching residual = 0.0 74.0% 74.0% 66.4% 66.4%
avg risk reduction 96.5% 96.5% 94.8% 94.8%
median risk reduction 100.0% 100.0% 100.0% 100.0%
records with injected deps 33.6% 33.6% 67.9% 67.9%
unique execution sequences 38 38 -- --

The injected deps rate nearly doubles in the hard split (33.6% to 67.9%) -- tighter budgets under high risk force more complex dependency chains.

These metrics are self-reported by the generator. They describe how the planner performs against its own scoring logic, not against externally validated PHI risk reduction.

Input risk distribution (default train)

Metric Value
mean 0.5191
stdev 0.2701
min / max 0.0502 / 0.9899

Input risk distribution (hard)

Metric Value
mean 0.8692
min / max 0.7500 / 0.9898

Quick Start

from datasets import load_dataset

ds = load_dataset("vkatg/dag_remediation_traces")
train = ds["train"]
test  = ds["test"]
import json

with open("data/train.jsonl") as f:
    records = [json.loads(line) for line in f]

planned = [r for r in records if r["status"] == "plan_ready"]
print(len(planned), "planned records")
print("avg residual risk:", sum(r["estimated_residual_risk"] for r in planned) / len(planned))

Schema

Field Type Description
patient_id string Synthetic patient identifier (P-000000 format)
input.risk_score float Re-identification risk score (0.0 to 1.0)
input.retok_prob float Re-tokenization probability (influences retokenize_text scoring)
input.active_modalities list Active data modalities: text, image, audio, ehr
input.budget float Total cost budget for the remediation plan
plan list Ordered list of actions with priority, cost, risk_delta, deps
trace.selected_actions list Actions chosen by the scorer before dependency injection
trace.injected_deps list Dependencies automatically injected to satisfy action prerequisites
trace.topo_order list Final topological execution order
trace.score_breakdown dict Per-action priority scores
estimated_residual_risk float Risk remaining after all actions execute
total_cost float Sum of costs across all actions including injected deps
status string plan_ready or below_threshold

Action Catalog (14 actions)

Action Cost Risk Reduction Modalities
mask_direct_id 0.05 0.35 text, image
generalize_dob 0.03 0.12 text, ehr
suppress_geo 0.04 0.10 text, ehr
redact_image_face 0.12 0.20 image
redact_image_tag 0.08 0.14 image
strip_audio_voice 0.15 0.18 audio
anon_audio_content 0.10 0.12 audio
drop_rare_code 0.06 0.09 ehr
k_anon_table 0.20 0.22 ehr, text
perturb_numerics 0.07 0.07 ehr
retokenize_text 0.09 0.11 text
cross_modal_unlink 0.18 0.28 all
federated_noise 0.25 0.15 all
audit_log_purge 0.02 0.03 all

Dependency graph

Some actions require others to run first:

  • retokenize_text requires mask_direct_id
  • k_anon_table requires generalize_dob, suppress_geo, drop_rare_code
  • cross_modal_unlink requires redact_image_face, strip_audio_voice, mask_direct_id
  • federated_noise requires k_anon_table, cross_modal_unlink
  • audit_log_purge requires federated_noise
  • redact_image_tag requires redact_image_face
  • anon_audio_content requires strip_audio_voice

When a selected action has unmet dependencies, those dependencies are injected automatically. 33.6% of planned records include at least one injected dependency.

Action frequency (train)

Action Occurrences
mask_direct_id 4,208
generalize_dob 4,175
suppress_geo 3,312
cross_modal_unlink 1,388
retokenize_text 1,156
redact_image_tag 950
audit_log_purge 352
redact_image_face 306
strip_audio_voice 233
drop_rare_code 189

Sample Record

{
  "patient_id": "P-000001",
  "input": {
    "risk_score": 0.9051,
    "retok_prob": 0.2147,
    "active_modalities": ["ehr"],
    "budget": 0.86
  },
  "plan": [
    {"action": "generalize_dob", "priority": 8.1366, "cost": 0.03, "risk_delta": 0.12, "deps": []},
    {"action": "suppress_geo",   "priority": 5.0854, "cost": 0.04, "risk_delta": 0.10, "deps": []},
    {"action": "cross_modal_unlink", "priority": 3.1642, "cost": 0.18, "risk_delta": 0.28, "deps": ["redact_image_face", "strip_audio_voice", "mask_direct_id"]}
  ],
  "trace": {
    "selected_actions": ["generalize_dob", "suppress_geo", "cross_modal_unlink", "drop_rare_code", "audit_log_purge", "k_anon_table"],
    "injected_deps": ["redact_image_face", "strip_audio_voice", "mask_direct_id", "federated_noise"],
    "topo_order": ["generalize_dob", "suppress_geo", "redact_image_face", "strip_audio_voice", "mask_direct_id", "cross_modal_unlink", "drop_rare_code", "federated_noise", "audit_log_purge", "k_anon_table"],
    "score_breakdown": {"generalize_dob": 8.1366, "suppress_geo": 5.0854, "cross_modal_unlink": 3.1642}
  },
  "estimated_residual_risk": 0.0,
  "total_cost": 1.1,
  "status": "plan_ready"
}

Planner Logic

The scorer ranks actions by a return-on-investment signal:

score = (reduces * modality_weight + retok_bonus) / base_cost * log(1 + risk * 5)

modality_weight is sampled per record (0.8 to 1.5) for a random subset of active modalities, reflecting variable sensitivity across data types. retok_bonus (+0.15) fires for retokenize_text when retok_prob > 0.55.

Selection adds actions greedily by score until the budget is hit or the risk reduction target is met (target = max(0, risk - 0.20), minimum spend 0.10). Dependency injection runs after selection to ensure topological validity.

Reproduce

python generate_dag_traces.py

The default train/test splits are deterministic at seed 7. The hard split uses seed 42 with risk sampled from [0.75, 0.99] and budget from [0.30, 0.60].

Related Models

Model Role
vkatg/exposureguard-dagplanner Model trained/evaluated on this dataset
vkatg/exposureguard-fedcrdt-distill Produces risk_score and retok_prob inputs
vkatg/exposureguard-dcpg-encoder Alternative source of risk_score

Related Datasets

Citation

@misc{ganti2025exposureguard,
  title        = {ExposureGuard: Cross-Modal PHI Re-identification Risk Scoring with DCPG and Federated CRDT Distillation},
  author       = {Ganti, Venkata Krishna Azith Teja},
  year         = {2025},
  doi          = {10.5281/zenodo.18865882},
  howpublished = {\url{https://huggingface.co/vkatg}}
}

License

MIT. Fully synthetic data. Contains no real patient information.

README history 10 revisions

Every night we snapshot the README of every dataset in the catalog. When the SHA changes we archive the new version and diff it against the last. This is the evolving thought record of the alignment-data field: what the author decided to say about the corpus, and how that framing shifted over time.

  1. 2026-03-13Update README.mdc326af79.7 KB
    Loading...
  2. 2026-03-13Upload 3 filese1923583.9 KB
    Loading...
  3. 2026-03-13Upload generate_dag_traces.py0a4f45e3.9 KB
    Loading...
  4. 2026-03-13Delete generate.pydfbc3b13.9 KB
    Loading...
  5. 2026-03-10Update README.md95a501d3.9 KB
    Loading...
  6. 2026-03-10Rename train.jsonl to data/train.jsonl1fa65b93.9 KB
    Loading...
  7. 2026-03-10Rename test.jsonl to data/test.jsonl56b62953.9 KB
    Loading...
  8. 2026-03-10Upload 3 files1eb5b5c3.9 KB
    Loading...
  9. 2026-03-10Update README.mdc1f439f3.9 KB
    Loading...
  10. 2026-03-10initial commitdc12d2024 B
    Loading...

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Related datasets same stage · misc

Read further

Metadata

License
mit

Repository

Download the dataset

0

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python)
from datasets import load_dataset
ds = load_dataset("vkatg/dag_remediation_traces")
Raw snapshot (Python)
from huggingface_hub import snapshot_download
snapshot_download(repo_id="vkatg/dag_remediation_traces", repo_type="dataset")
git clone (requires git-lfs)
git clone https://huggingface.co/datasets/vkatg/dag_remediation_traces
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration