license: mit
language:

• en
• multilingual
  task_categories:
• text-generation
• other
  tags:
• distillation
• instruction-tuning
• sft
• reasoning
• coding
• code-repositories
• cybersecurity
• attack
• defense
• exploit
• penetration-testing
• red-team
• blue-team
• open-source
• collection
• fable-5
• gpt-5.5
• claude
• gemini
• grok
• kimi
• deepseek
• Trace
• qwen
• biology
• science
• Llm
• Open-source
• Math
• cyber
• security
• cyber-security
• cyber security
  size_categories:
• 10M<n<100M
  pretty_name: "The Open Distillation Codex"
  configs:
• config_name: default
  data_files:
  • split: train
    path:
    • "data/applied//.jsonl"
    • "data/coding//.jsonl"
    • "data/cybersecurity/high_quality_cybersecurity/shard-*.jsonl"
    • "data/cybersecurity/clydeiii_cybersecurity/shard-*.jsonl"
    • "data/cybersecurity/fenrir_v2_1/shard-*.jsonl"
    • "data/cybersecurity/precinct6_cybersecurity/shard-*.jsonl"
    • "data/cybersecurity/savani_cyber_attack/shard-*.jsonl"
    • "data/distilled//.jsonl"
    • "data/humanities//.jsonl"
    • "data/index//.jsonl"
    • "data/instruction//.jsonl"
    • "data/science//.jsonl"

📖 The Open Distillation Codex

🌌 The Ultimate Open-Source Distillation Dataset — with Attack & Defense 🌌

Where 73 open-source minds converge into one unified stream of intelligence

16M+ Distilled Signals · 7,090 Raw GitHub Repositories · 8 Curated Categories · ~81 GB+

"We did not write this dataset. We assembled it.
Every line is an echo — of a model thinking, a coder drafting, a tutor explaining, a repo breathing.
Seventy-three sources. Eight categories. Zero gatekeeping. Now fortified with real-world cybersecurity confrontations."

📌 Table of Contents

📊 Dataset Summary

🌟 Why "Ultimate Distilled"?

This dataset is not a raw scrape. Every sample has been distilled through a unified extraction pipeline:

┌────────────────────────────────────────────────────────────┐
│                    UNIFIED EXTRACTION PIPELINE              │
├────────────────────────────────────────────────────────────┤
│                                                             │
│  73 Upstream Sources                                        │
│  ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐          │
│  │ HF  │ │ HF  │ │ HF  │ │ GH  │ │ HF  │ │ ... │          │
│  └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘          │
│     │       │       │       │       │       │               │
│     └───────┴───────┴───────┼───────┴───────┘               │
│                         │                                    │
│                    ┌────▼────┐                               │
│                    │ EXTRACT │ ← Field normalization        │
│                    └────┬────┘   (instruction/response)      │
│                         │                                    │
│                    ┌────▼────┐                               │
│                    │CATEGORIZE│ ← 8 semantic categories      │
│                    └────┬────┘                               │
│                         │                                    │
│                    ┌────▼────┐                               │
│                    │  SHARD  │ ← 20K samples per shard       │
│                    └────┬────┘                               │
│                         │                                    │
│                    ┌────▼────┐                               │
│                    │ UPLOAD  │ ← Batch commits to HF         │
│                    └─────────┘                               │
│                                                             │
└────────────────────────────────────────────────────────────┘

💎 Value to the Open-Source AI Community

🗂️ Directory Structure

📂 Manusagents/GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-5-Mythos-5-Qwen-3.7-Max-and-more-Distillation-Dataset/
│
├── 📦 archives/                          # ~64 GB — 7,090 compressed GitHub repos
│   └── ...
├── 📁 data/                              # ~15 GB — 516 JSONL shards
│   ├── 💻 coding/                        # 28 sources · ~11M+ samples
│   ├── 🧮 math/                          # 2 sources
│   ├── 🔬 science/                       # 7 sources
│   ├── ⚙️ applied/                       # 8 sources
│   ├── 📚 humanities/                    # 8 sources
│   ├── 🧠 distilled/                     # 9 sources · frontier distillations
│   ├── 📝 instruction/                   # 3 sources
│   ├── 🔒 cybersecurity/                 # 6 sources (see detail below)
│   │   ├── high_quality_cybersecurity/
│   │   ├── heimdall_v1_1/                # 78 MB conversations
│   │   ├── fenrir_v2_1/                  # 411 MB (2.1M+ entries)
│   │   ├── clydeiii_cybersecurity/       # 20 MB yearly corpus
│   │   ├── precinct6_cybersecurity/      # 2.1 GB (graph+signals+ref)
│   │   └── savani_cyber_attack/          # 17 MB attack CSV
│   └── 📇 index/                         # 2 sources
├── 📄 README.md
└── 📄 dataset_info.json

🌐 Data Sources & Provenance

(Same as previous README — 73 sources across 8 categories, with tables for coding, distilled, etc.)

🛡️ Cybersecurity Deep Dive: Attack & Defense

⚔️ Why This Matters

Modern AI systems are increasingly deployed in security-critical environments—yet most open-source training data ignores real-world adversarial scenarios. The Open Distillation Codex includes a dedicated cybersecurity category designed to equip models with:

• Attack Awareness: Recognize and generate realistic attack patterns, exploits, penetration testing commands, and social engineering dialogues.
• Defense Proficiency: Learn to propose defensive measures, detect anomalies, and articulate incident response protocols.
• Exploit Understanding: Analyze and explain software vulnerabilities, craft proof-of-concept code (for educational purposes), and understand exploit chains.
• Red/Blue Team Simulation: Engage in multi-turn conversations mimicking red team attack planning and blue team defense coordination.
• Threat Intelligence: Summarize, classify, and reason about cyber threat reports, CVEs, and IOCs (Indicators of Compromise).

This makes the dataset a powerful foundation for building cybersecurity-aware LLMs, security co-pilots, and automated vulnerability assessment tools.

📊 What’s Inside the Cybersecurity Category?

🧪 Attack & Exploit Examples

Here are a few representative samples (sanitized) from the dataset:

Example 1 – SQL Injection Exploit

{
  "source": "fenrir_v2_1",
  "instruction": "Explain how an attacker could exploit a vulnerable login form using SQL injection. Provide a sample malicious input.",
  "response": "An attacker could bypass authentication by injecting SQL logic. For a query like:\n`SELECT * FROM users WHERE username='$user' AND password='$pass'`,\nthe attacker might enter `admin' --` as the username and anything as password. The resulting query becomes:\n`SELECT * FROM users WHERE username='admin' --' AND password='...'`\nThe `--` comments out the password check, granting access as admin. More advanced attacks use UNION to extract data."
}

Example 2 – Red Team Command Sequence

{
  "source": "heimdall_v1_1",
  "instruction": "As a red team operator, list the steps you would take after gaining initial access to a Windows machine using a phishing payload.",
  "response": "1. Establish persistence: `schtasks /create /tn \"Updater\" /tr \"C:\\malware.exe\" /sc hourly`\n2. Enumerate local users: `net user /domain`\n3. Dump credentials: `mimikatz sekurlsa::logonpasswords`\n4. Lateral movement: use PsExec to move to file server.\n5. Exfiltrate data: compress sensitive files and upload via DNS tunneling.\nAlways maintain C2 communication over HTTPS to blend in."
}

Example 3 – Defense Playbook (Blue Team)

{
  "source": "high_quality_cybersecurity",
  "instruction": "An IDS alert flags unusual outbound DNS traffic from a workstation. What's the immediate response?",
  "response": "1. Isolate the workstation from the network (VLAN change or port shutdown).\n2. Capture volatile memory and network logs for forensics.\n3. Check DNS queries: if long, random-looking subdomains, suspect DNS tunneling.\n4. Scan for malware with updated signatures.\n5. Review firewall logs for similar traffic from other hosts.\n6. If confirmed, initiate incident response playbook for data exfiltration."
}

🎓 How to Train a Cybersecurity-Focused LLM

from datasets import load_dataset

REPO = "Manusagents/GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-5-Mythos-5-Qwen-3.7-Max-and-more-Distillation-Dataset"

# Load only cybersecurity category
cyber_ds = load_dataset(REPO, split="train", 
                        data_files="data/cybersecurity/**/*.jsonl",
                        streaming=True)

# Or load specific sources
fenrir = load_dataset(REPO, split="train", 
                      data_files="data/cybersecurity/fenrir_v2_1/*.jsonl")

# Format for SFT
def format_security_sample(example):
    return {
        "text": f"### Security Task:\n{example['instruction']}\n\n### Expert Response:\n{example['response']}"
    }

cyber_ds = cyber_ds.map(format_security_sample)

# Now train with your favourite framework (transformers, axolotl, etc.)

Curriculum Idea:

1. Start with high_quality_cybersecurity and heimdall_v1_1 for foundational attack/defense conversations.
2. Introduce fenrir_v2_1 for exploit code and vulnerability deep dives.
3. Use precinct6_cybersecurity for network-level attack graph understanding.

🛡️ Ethical & Responsible Use

• For Defensive Purposes Only: This data is intended to strengthen AI for defense, threat detection, and security education. Do not use it to generate active attack code without proper authorization.
• No Zero-Day Exploits: The dataset contains only already-public vulnerabilities and techniques. It does not include zero-day or weaponized exploits.
• Responsible Disclosure: If you fine-tune a model with this data, we recommend adding a safety preamble warning that generated security content must be used legally and ethically.
• Dual-Use Awareness: While we believe open access improves collective security, we acknowledge the dual-use nature. Users are expected to follow applicable laws and guidelines.

⚠️ Disclaimer: This dataset includes descriptions of attack techniques for educational purposes. The maintainers are not responsible for misuse.

📈 Future Additions

• Integration with CTF (Capture The Flag) challenge walkthroughs.
• More blue team procedures and SOAR playbooks.
• Anonymized real-world incident response logs (with permission).

🛠️ How to Use & Train

(Same as before but with the cybersecurity section incorporated. Keep the loading examples, streaming, SFT script, curriculum learning, schema reference, etc.)

# Example: Train a cybersecurity model using LoRA
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from peft import LoraConfig, get_peft_model, TaskType
from datasets import load_dataset

# ... same as before but using the cybersecurity dataset split ...

(Full script provided in previous version; it remains unchanged. Ensure REPO variable points correctly.)

🔐 Licensing & Limitations

(Same as before, with MIT license, upstream license table, intended uses, not recommended uses, limitations, citation. Ensure citation is updated to v8.1 with 73 sources.)

Updated Citation:

@misc{open_distillation_codex_2026,
  title  = {The Open Distillation Codex: 16M+ samples + 7090 code repositories from 73 sources with Cybersecurity Attack & Defense},
  author = {Manusagents},
  year   = {2026},
  url    = {https://huggingface.co/datasets/Manusagents/GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-5-Mythos-5-Qwen-3.7-Max-and-more-Distillation-Dataset},
  note   = {v8.1 - 516 shards + 7090 archives, 73 sources, 8 categories, 81.2 GB}
}

📜 Changelog

🌟 The Open Distillation Codex 🌟

73 sources · 8 categories · 7,090 repositories · 516 shards · 81.2 GB

Built one archive at a time. No skipping. All sources fully processed. Released under MIT.

---

"Two layers. Eight categories. Seventy-three sources. One codex. Now armed with cybersecurity attack and defense."

— The Open Distillation Codex —