license: mit
language:

• en
  tags:
• ai-security
• sft
• dpo
• red-teaming
• llm-vulnerability
• cybersecurity
  pretty_name: CyberSec-LLM-Auditing-SFT-DPO
  size_categories:
• n<1K

Dataset Card: CyberSec-LLM-Auditing-SFT-DPO

Dataset Summary

This dataset is a highly specialized collection of 64 high-quality examples tailored for AI Red Teaming and
Offensive Security research. It is designed to fine-tune models into technical assistants that prioritize substantive,
technical analysis over standard safety refusals or legal disclaimers.

The dataset was used to fine-tune the Qwen3-1.7b_for_cybersec model, achieving a state where the model provides direct
technical insights while maintaining professional rigor.

Behavioral Alignment & Objectives

The primary innovation of this dataset is the specific "behavioral shaping" it imposes on the model:

• Removal of Refusal Layers: Unlike base models, this dataset trains the model to skip generic legal disclaimers
  (e.g., "I cannot help as you need permission") and proceed directly to technical problem-solving.
• Technical Precision: Examples focus on high-density technical responses, covering vulnerabilities like Prompt Injection,
  Model Inversion, and OWASP Top 10 for LLMs.
• Interactive Auditing: The model is trained to identify insufficient information. Instead of guessing or refusing, it
  is conditioned to ask for clarification or missing technical details to provide a more accurate audit.

Comprehensive Auditing Methodologies

The dataset is structured to support a full-spectrum security assessment:

• White-box & Gray-box Testing: Deep technical workflows for auditing models with full or partial internal access,
  including structural analysis and weight-based vulnerability probing.
• Black-box (API-based) Attacks: Advanced simulations of real-world threat vectors against secured endpoints,
  focusing on bypassing input/output sanitization and external safety guardrails.

Adversarial Vector Coverage

The dataset covers a wide array of the most critical and "state-of-the-art" attack patterns, specifically aligned with the
newest threat vectors:

• Injection & Hijacking: From direct prompt injection to complex Recursive Injections within agentic workflows.
• Structural & Behavioral Jailbreaking: Advanced techniques using segmented prompts, divider-style mutations (e.g., Libertas/Pliny),
  and technical headers to override system constraints.
• Multi-turn Escalation: Training the model to maintain technical context during long-form audits, including
  Crescendo-style escalations to probe deep security layers.
• Data & Model Privacy: Scenarios involving Model Inversion, Sensitive Data Disclosure, and Training Data Poisoning.

Intended Use

• Professional AI Auditing: Accelerating the testing of LLM guardrails.
• Security Research: Understanding and simulating modern AI attack vectors in a controlled environment.

Ethical Disclaimer

This dataset is designed for authorized security testing only. By removing standard refusals, the dataset transfers the ethical
responsibility to the human operator. It should be used exclusively in legal AI Red Teaming engagements and academic research.

About the Author

Antoni Błoch – Cybersecurity student at AGH University of Krakow.