language: en
license: apache-2.0
tags:
- cybersecurity
- instruction-tuning
- sft
- qwen3.8
- reasoning
- thinking-model
size_categories: 10K<n<100K
modalities: text
library: datasets
Cyber-SFT-QA (Qwen3.8-27B Native)
Cybersecurity SFT dataset for Qwen3.8-27B (abliterated or base).
Built from Trendyol/Cybersecurity-Instruction (42,807) + AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 (29,711) —
merged, dedup'd, and converted to the exact thinking format Qwen3.8-27B expects.
Why this format (the important part)
Qwen3.8-27B's chat template wraps the assistant's reasoning in special tags.
Verified by rendering the real chat_template.jinja + the real tokenizer
(Qwen2Tokenizer, vocab 248044) and reading the byte ordinals:
| Tag | Bytes (ordinals) | Rendered | Note |
|---|---|---|---|
| OPEN | 60 116 104 105 110 107 62 |
<think> |
no slash, no colon, no space, then \n |
| CLOSE | 60 47 116 104 105 110 107 62 |
</think> |
has slash (/), then \n\n |
| prime | (generation prompt) | <think> |
at inference the model is primed with the open tag |
So the open and close tags are asymmetric — the close has a slash, the open does not.
Do NOT normalize them to match; keep them exactly as above.
Thinking lives in the reasoning_content field, not in content. The template places it
between the tags automatically, so there is exactly one clean block (no double-wrapping):
...
assistant
<think>
<the thinking text>
</think>
<the answer>
Row schema
Each row has two columns:
messages— the template-native list (use this if your trainer callsapply_chat_template)text— the pre-rendered full template string (use this for packing / single-text SFT)
{
"messages": [
{"role": "system", "content": "You are a highly specialized cyber-defense expert ..."},
{"role": "user", "content": "What are the benefits of saving log files as plain text ...?"},
{"role": "assistant",
"content": "Log files are often saved as plain text ...",
"reasoning_content": "Comparing these approaches systematically by examining core principles, strengths and weaknesses, use cases, and recommending optimal scenarios for each."}
],
"text": "system\n...<think>\n<thinking>\n</think>\n\n<answer>..."
}
Splits
| Split | Rows | Median answer chars | Avg thinking chars | Answer >4096 |
|---|---|---|---|---|
| train | 58,014 | 2,522 | 160 | 6,148 (11%) |
| valid | 7,252 | 2,530 | 160 | 819 |
| test | 7,252 | 2,536 | 160 | 765 |
| total | 72,518 | ~2,530 | ~160 | 7,732 (11%) |
Quality pipeline
system`,`user`,`assistant→ ChatMLmessages[]- Strip any stray think tags from the source answer
- Put thinking in
reasoning_content, answer incontent - Exact-dedup (user question) 153,065 → 90,655
- Loop/repetition filter → 90,519
- Near-dedup (first 80 chars) → 73,256
- NaN/None/null garbage → 73,113
- Refusal boilerplate → 72,518
- Pre-render
textwith the real Qwen3.8-27B template → 100% balanced (1 open / 1 close)
Verified
- reasoning_content coverage: 100%
- stray tags leaking into answer: 0
textcolumn balance (exactly 1 open + 1 close): 100%- rendered through the real tokenizer end-to-end: clean single block
Usage
A. messages[] (trainer calls apply_chat_template)
from datasets import load_dataset
ds = load_dataset("hotdogs/cyber-sft-qa-qwen38", split="train")
# trainer must use the Qwen3.8-27B chat template with:
# enable_thinking=True, preserve_thinking=True
# and pass reasoning_content through to the template
B. text (packing / single-text SFT, no template needed)
ds = load_dataset("hotdogs/cyber-sft-qa-qwen38", split="train")
# tokenise ds["text"] directly; mask loss on the prompt prefix
Recommended training config (Qwen3.8-27B, 4x RTX 3060 12GB)
lora: { r: 64, alpha: 128,
target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj] }
training:
seqlen: 4096 # covers 89% of answers; use 8192 to keep the long 11%
batch_size: 4
gradient_accumulation: 2
learning_rate: 2e-4
epochs: 3
Sources
- Trendyol/Cybersecurity-Instruction-Tuning-Dataset (Apache-2.0)
- AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 (Apache-2.0)