license: mit
task_categories:

• text-to-image
• reinforcement-learning
  tags:
• retro-pixel-art
• svg
• reinforcement-learning
• representation-engineering
• qwen3
• pixel-art

RetroIcon-RL: 32x32 Retro Pixel-Art Icon Generation with Deterministic Verification

This repository implements RetroIcon-RL — training and evaluating small code models with Reinforcement Learning (RL) and programmatic verification to generate consistent, crisp retro pixel-art icon packs from natural language prompts using sharp SVG block geometry.

🎯 Phase 1 — The 32x32 Task Specification

• Canvas: Exactly $32 \times 32$ integer grid (viewBox="0 0 32 32").
• Palette: Fixed 8-Color Retro Palette:
  • BLACK: #000000 | DARK_GRAY: #2B2D42 | WHITE: #FFFFFF | CYAN: #00E5FF
  • PURPLE: #9D4EDD | GREEN: #00E676 | YELLOW: #FFD600 | RED: #FF1744
• Geometry Rules: Strict integer <rect> pixel blocks. No bezier <path>, no <circle>, no gradients, no anti-aliasing.

🛠️ Phase 2 — Deterministic Rule-Based Validator & Pure-Python Renderer

The pipeline prompt ➔ model ➔ structured SVG ➔ validator ➔ pixel-perfect PNG enforces 6 automated checks:

1. XML/SVG parse tree well-formedness.
2. Exact $32 \times 32$ dimensions.
3. 100% adherence to the fixed 8-color palette.
4. All coordinates within $[0, 32]$.
5. Integer-only rectangular geometry without bezier smoothing.
6. Minimum icon density (>= 3 distinct geometry blocks).

🚀 The Master V2 Pipeline (SFT + GRPO)

We implemented Surya Narreddi's Pairwise RL methodology to train Qwen3-8B on an NVIDIA H100.

1. SFT Warm-Start (120 Gold Experts)

We synthesized 120 canonical 32x32 pixel-art icons using Gemini 2.5 Flash (data/expert_expanded_100.jsonl). Fine-tuning on this dataset gave the base model the 2D spatial coordinate grammar needed to place pixels accurately.

2. Surya-Style Pairwise GRPO Training

Using the SuryaRewardEngine, we abandoned absolute 1-to-10 scoring. For every generation step, Qwen generated $G=4$ rollouts. The engine pitted them in blind 1v1 matchups against the Gold Reference pool, judged by Gemini 2.5 Flash on silhouette clarity and aesthetic polish.

$$R(x) = \text{validity_gate}(x) \times \Big( 0.05 \times \text{efficiency_bonus}(x) + 0.95 \times \text{pairwise_win_rate}(x, \text{Pool}) \Big)$$

Results

The final policy achieved 100% syntactic validity and beat or tied the Gemini 2.5 Flash Gold standard on highly complex visual metaphors (e.g. mushroom fungi, spell book, hard drive, pocket compass, dynamite fuse, store storefront, camera photo).

🎨 Experiment 2: Paint-with-Code (Qwen2.5-Coder-1.5B)

We also ran a pure Direct RL (GRPO) experiment on Qwen2.5-Coder-1.5B to draw watercolor hibiscus flowers using Python matplotlib.patches.

• Result: The model learned 100% executable Python syntax in just 10 steps. However, without an SFT warm-start, it hit an aesthetic plateau, generating structurally valid but visually crude geometric blobs, proving that RL requires a strong structural prior for complex creative tasks.

🚀 Reproduction

# 1. Clone repository
git clone https://huggingface.co/datasets/mayank-dubey-ai/retro-icon-rl
cd retro-icon-rl

# 2. Run the full Master V2 Pipeline (SFT + GRPO + Showcase)
python3 train_master_v2.py