base_model: orcarouter/Qwen3.8-27B-Uncensored-FP8
library_name: transformers
pipeline_tag: image-text-to-text
license: other
language:
- en
- fr
tags: - unsloth
- lora
- qlora
- nuclei
- image-text-to-text
- vision
- video
model-index:
- name: Qwen3.8-27B-Uncensored-FP8-nuclei
results:- task:
type: text-generation
name: Causal language modeling
dataset:
name: nuclei SFT holdout
type: dataset_nuclei.jsonl
metrics:- type: loss
value: 14.090689
name: eval_loss
- type: loss
- task:
Qwen3.8-27B-Uncensored-FP8-nuclei
Instruction-tuned nuclei assistant to generate from CVE / exploit a nuclei script
| Base model | orcarouter/Qwen3.8-27B-Uncensored-FP8 |
| Domain | nuclei |
| Method | LoRA / QLoRA (Unsloth) · rank 8 · α 16 |
| Quantization at train | bf16 LoRA |
| Context | 2048 tokens |
| Dataset | dataset_nuclei.jsonl · train 919 / eval 10 |
| GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition (95.0 GiB) |
| Wall time | 16.7 min |
| Modalities kept | vision, video |
LoRA/QLoRA fine-tune of the base model on a nuclei SFT dataset.
What changed vs the reference
Reference = the published base checkpoint orcarouter/Qwen3.8-27B-Uncensored-FP8, plus the first in-run loss (LoRA ≈ 0 at step 0).
| Metric | Reference (base / first log) | This fine-tune | Δ |
|---|---|---|---|
| Train loss (first → last logged) | 14.3511 | 14.2714 | -0.6% |
| Train loss (best) | — | 14.1145 | — |
| Eval loss (holdout, first → last) | 14.0907 | 14.0907 | +0.0% |
The first logged train loss is the closest in-run proxy for the base model (LoRA starts near zero). Option F, when executed, adds an independent holdout comparison against the frozen merged base.
Training data
- File:
dataset_nuclei.jsonl - Split:
0.01holdout, seed 42 - Format: chat-templated SFT (
messages/instruction+output/### Instruction+### Response)
Training procedure
| Hyperparameter | Value |
|---|---|
| Epochs | 1 |
| Learning rate | 0.0002 |
| Warmup ratio | 0.05 |
| Device batch | 4 |
| Grad accum | 2 |
| Effective batch | 8 |
| Optim | adamw_8bit |
| Packing | True |
| LoRA targets | ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'out_proj', 'gate_proj', 'up_proj', 'down_proj'] |
Intended use
Domain Q&A and drafting in the training domain.
Out of scope: Anything outside the training domain or that requires certification.
Multimodal
Kept towers: vision, video. Vision/audio layers were frozen during text SFT (vision=False, audio=False). Load the merged Transformers folder (or GGUF + mmproj) to keep image / video / audio.
How to use
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "nico248000000000/Qwen3.8-27B-Uncensored-FP8-nuclei"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
messages = [{"role": "user", "content": 'How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list.'}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(inputs, max_new_tokens=256)[0], skip_special_tokens=True))
Limitations
- Domain shift: quality drops outside the SFT topics.
- Eval above is holdout loss (and optional targeted checks). It is not a public leaderboard.
- The base model license and acceptable-use policy still apply.
License
other — inherit and respect the license of orcarouter/Qwen3.8-27B-Uncensored-FP8.