license: cc-by-4.0
pretty_name: The AI Slop Field Guide
language:

• en
  task_categories:
• text-classification
  tags:
• ai-detection
• writing-style
• guardrails
• evaluation
• style-transfer
• constitutional-ai
  size_categories:
• n<1K
  configs:
• config_name: default
  data_files:
  • split: train
    path: data/tells.jsonl

The AI Slop Field Guide

45 labeled patterns of unedited language model output, with a scoring rubric and a
ready-to-paste constraint prompt.

A free asset from Consumer Dividends.

More than half of what you read online now started as language model output. Graphite sampled
65,000 English-language articles published between January 2020 and May 2025 and found AI-generated
pieces crossed 50 percent of new content in November 2024, settling around 52 percent. That is not
automatically a problem. AI-assisted writing helps non-native speakers get ideas across clearly,
and it gives people without an editor a way to sound coherent under deadline.

The problem is generated text passed off as original thought by people charging for original
thought. If someone wants your attention or your money on the strength of their thinking, you
should be able to tell whether the thinking is theirs. This dataset is the machine-readable version
of how to tell.

There is a second use, and it is the one we care more about. If you are building something that
writes, you can load these 45 patterns as constraints and stop your product from adding to the
pile.

What is in here

Schema

{
  "id": "AR-04",
  "category": "argumentation",
  "name": "Label-level citation",
  "definition": "Evidence attributed to a category instead of a source...",
  "mechanism": "The model produces the shape of a citation because citations are common...",
  "slop_example": "Research suggests that remote workers are more productive.",
  "clean_rewrite": "Bloom's 2015 Ctrip experiment found a 13 percent productivity gain among 249...",
  "severity": 5,
  "weight": 5,
  "detection_type": "lexical",
  "detection_patterns": ["\\b(research|studies|data) (suggests?|shows?)\\b"],
  "count_mode": "absolute",
  "threshold_per_1000": 0.0,
  "applies_to": "any",
  "false_positive_note": "Acceptable as a summary sentence immediately followed by the citation.",
  "guide_section": "Argumentation Tells"
}

detection_type is one of lexical, structural, or judgment. Eleven of the 45 are judgment
calls that no regex reaches, and fifteen carry no usable pattern at all. Those are the ones worth
spending a model call on.

Categories

Using it

Load the taxonomy:

from datasets import load_dataset

tells = load_dataset("ConsumerDividends/ai-slop-field-guide", split="train")
argument_tells = tells.filter(lambda t: t["category"] == "argumentation")

Constrain a model you are building:

from huggingface_hub import hf_hub_download

path = hf_hub_download("ConsumerDividends/ai-slop-field-guide",
                       "configs/system_prompt.md", repo_type="dataset")
system_prompt = open(path).read()

Score a draft:

python scripts/score.py draft.md
python scripts/score.py draft.txt --plaintext   # venue does not render markdown
cat draft.txt | python scripts/score.py

Build a judge prompt for the eleven judgment tells:

judgment = [t for t in tells if t["detection_type"] == "judgment"]
prompt = "\n".join(
    f"{t['id']} {t['name']}: {t['definition']}\nException: {t['false_positive_note']}"
    for t in judgment
)

Scoring

score = min(100, round(100 * points / 60)), where each triggered pattern contributes
weight * min(1 + 0.5 * (n - 1), 2.5). Repeats give diminishing returns so one quirk cannot carry
the result. Sixty is a calibration constant, not a discovered value. Recalibrate it on your corpus
and write down what you used.

Four overrides run after the formula. Chat-turn residue floors the score at 90, because
"Certainly! Here's a draft" is addressed to whoever ran the prompt rather than to the reader.
Passages under 300 words get flagged low-confidence. A lexical-only result caps at 45, since
vocabulary is the easiest signal to launder. Three or more argumentation hits floor at 55, since
those survive an editing pass designed to defeat everything else.

Bands: 0-14 clean, 15-29 light assistance, 30-49 mixed, 50-74 likely generated, 75-100 raw slop.

The scorer strips code fences, blockquotes, inline code, and short quoted spans before it counts
anything. Without that, any document criticizing these patterns scores as an instance of them. The
test suite checks this by scoring the field guide against its own rubric: 7 with quote stripping,
100 without.

Limits, stated plainly

This measures resemblance, not authorship. No published detector reliably identifies who wrote
something, and this one does not either. A high score is a reason to verify claims. It is not a
finding of fact, and it should never be the sole basis for an academic or employment decision.

Non-native English writers get hit hardest. Formal register, low sentence-length variance, and
heavy discourse markers all characterize competent second-language academic writing, and several
word-choice and sentence-structure entries will fire on it. This is the failure mode every
stylometric approach shares and the most likely way this dataset gets misused. If you are building
on it, handle this explicitly rather than hoping it does not come up.

The lexical entries have a shelf life. As word lists circulate, both models and writers route
around them. The structural and argumentation entries degrade more slowly because they follow from
how generation works rather than from which tokens it favors.

These patterns describe models as of mid-2026. Newer systems already produce fewer word-choice
entries and roughly as many argumentation entries, which is what the mechanics predict.

If you are going to use a model to write

Take configs/system_prompt.md and put it in front of your prompt. That is the whole ask.

Citation

@misc{consumerdividends2026slop,
  title  = {The AI Slop Field Guide},
  author = {Consumer Dividends},
  year   = {2026},
  note   = {Version 1.0.0},
  url    = {https://huggingface.co/datasets/ConsumerDividends/ai-slop-field-guide}
}

Sources

• Graphite, "More Articles Are Now Created by AI Than Humans" (2025). Sample of 65,000 English-language articles, January 2020 to May 2025.
• Thompson, Dhaliwal, Frisch, Domhan, Federico, "A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism", Findings of ACL 2024.

Licensed CC BY 4.0. Use it, fork it, sell what you build with it. Keep the attribution.