license: apache-2.0
language:
- en
tags: - code
- cybersecurity
- hacking
- crypto
- devops
- uncensored
- monico
pretty_name: Monico Training Data
size_categories: - 100B<n<1T
Monico Training Data
Official training corpus for Monico Model — a ground-up uncensored LLM.
Sources
Data is collected, filtered, and deduplicated via the pipeline at jaykk99/monico-model.
| Domain | Sources | Tokens (est.) |
|---|---|---|
| Offensive Security | Exploit-DB, CVE NVD, GitHub sec repos | ~20B |
| Hacking / OSINT | CTF writeups, forums, research papers | ~15B |
| Code (all langs) | The Stack v2, GitHub (permissive) | ~400B |
| Smart Contracts | Etherscan verified contracts, Solidity docs | ~10B |
| Crypto / DeFi | DeFi protocol docs, audit reports, MEV research | ~8B |
| DevOps | Docker Hub docs, K8s docs, Terraform examples | ~15B |
| App Building | Full-stack tutorials, architecture patterns | ~20B |
| Account Automation | Browser automation scripts, signup research | ~5B |
| Business Automation | Lead gen, outreach, workflow scripts | ~5B |
| General Knowledge | Books, Wikipedia, arXiv, reasoning datasets | ~200B |
Processing
See src/data/preprocess.py in the GitHub repo for full pipeline:
- HTML/PDF → clean text
- MinHash LSH deduplication (Jaccard threshold 0.8)
- Quality filtering (perplexity, length, code syntax)
- BPE tokenization (custom 65k vocab)
- Context packing (8192 token windows)
GitHub Import
git clone https://github.com/jaykk99/monico-model
cd monico-model/src/data
pip install -r requirements.txt
python collect_github.py --domains security hacking crypto devops
python collect_hf_datasets.py
python preprocess.py --output data/processed/