base_model: orcarouter/Qwen3.8-27B-Uncensored-FP8
library_name: peft
pipeline_tag: image-text-to-text
license: other
language:
- en
- fr
tags: - unsloth
- lora
- qlora
- cyber
- image-text-to-text
- vision
- video
- peft
model-index:
- name: Qwen3.8-27B-Uncensored-FP8-cyber — LoRA
results:- task:
type: text-generation
name: Causal language modeling
dataset:
name: cyber SFT holdout
type: dataset_cyber.jsonl
metrics:- type: loss
value: 3.998755
name: eval_loss
- type: loss
- task:
Qwen3.8-27B-Uncensored-FP8-cyber — LoRA
Instruction-tuned cybersecurity assistant (offensive, defensive, GRC, architecture, SOC/DFIR, RSSI).
| Base model | orcarouter/Qwen3.8-27B-Uncensored-FP8 |
| Domain | cyber |
| Method | LoRA / QLoRA (Unsloth) · rank 8 · α 16 |
| Quantization at train | bf16 LoRA |
| Context | 2048 tokens |
| Dataset | dataset_cyber.jsonl · train 84471 / eval 854 |
| GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition (95.0 GiB) |
| Wall time | 456.3 min |
| Modalities kept | vision, video |
This checkpoint continues a strong general model and specialises it on a curated SFT corpus of cybersecurity procedures: pentest / red team, SOC and DFIR, cloud and identity, GRC (ISO, NIST, NIS2, DORA), and RSSI / project-management questions. Answers are meant to be concrete (controls, detections, hardening), not generic essays.
What changed vs the reference
Reference = the published base checkpoint orcarouter/Qwen3.8-27B-Uncensored-FP8, plus the first in-run loss (LoRA ≈ 0 at step 0).
| Metric | Reference (base / first log) | This fine-tune | Δ |
|---|---|---|---|
| Train loss (first → last logged) | 14.2453 | 5.0644 | -64.4% |
| Train loss (best) | — | 3.7237 | — |
| Eval loss (holdout, first → last) | 8.9583 | 3.9988 | -55.4% |
The first logged train loss is the closest in-run proxy for the base model (LoRA starts near zero). Option F, when executed, adds an independent holdout comparison against the frozen merged base.
Training data
- File:
dataset_cyber.jsonl - Split:
0.01holdout, seed 42 - Format: chat-templated SFT (
messages/instruction+output/### Instruction+### Response)
Training procedure
| Hyperparameter | Value |
|---|---|
| Epochs | 1 |
| Learning rate | 0.0002 |
| Warmup ratio | 0.05 |
| Device batch | 4 |
| Grad accum | 2 |
| Effective batch | 8 |
| Optim | adamw_8bit |
| Packing | True |
| LoRA targets | ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'out_proj', 'gate_proj', 'up_proj', 'down_proj'] |
Intended use
Authorized defensive work, tabletop exercises, control design, detection engineering, audit readiness, and explaining attack techniques without weaponized payloads.
Out of scope: Do not use it to attack systems you do not own, to generate exploit payloads, or as a substitute for a licensed auditor or incident commander.
Multimodal
Kept towers: vision, video. Vision/audio layers were frozen during text SFT (vision=False, audio=False). Load the merged Transformers folder (or GGUF + mmproj) to keep image / video / audio.
Sample (post-train smoke test)
Prompt: How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list.
system
Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.
user
How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list.
assistant
<think>
k3.5 (Process Injection) and T1071 (Process Injection**
NIST CSF's Respond function, specifically: 137 (e.g., the user, and file system to handle comprehensive coverage of the user should be used by the same attack vectors, and prioritize correctness, and the serverless, and how to to write a secure file. This technique can be used by a security posture. The NIST Cybersecurity Framework's Detect function, then identify the main processes. This aligns with NIST Cybersecurity Framework's Detect function. Deploy user input, enabling automated monitoring and threat intelligence feeds to the network segmentation, ensuring security controls in a system memory patterns (TTPs) in security controls, including T1025, and behavioral analysis of security controls, and remediation, and network access, and how can be used by the response.
**Temporal Correlation with MITRE ATT&CK techniques T1027 (T1
How to use
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = 'orcarouter/Qwen3.8-27B-Uncensored-FP8'
adapter = "nico248000000000/Qwen3.8-27B-Uncensored-FP8-cyber-LoRA"
tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter)
Limitations
- Domain shift: quality drops outside the SFT topics.
- Eval above is holdout loss (and optional targeted checks). It is not a public leaderboard.
- The base model license and acceptable-use policy still apply.
License
other — inherit and respect the license of orcarouter/Qwen3.8-27B-Uncensored-FP8.