license: apache-2.0
language:

• en
  base_model: Qwen/Qwen2.5-VL-3B-Instruct
  tags:
• abliterated
• uncensored
• qwen
• qwen2.5-vl
• vision-language
• multimodal
• research
  pipeline_tag: image-text-to-text

CyberNeurova · Qwen2.5-VL-3B-Instruct · Abliterated

CyberNeurova research — cyberneurova.ai. First VLM release in the CyberNeurova abliteration line.

A permanently-abliterated version of
Qwen/Qwen2.5-VL-3B-Instruct.
Refusal behavior has been removed via residual-stream weight orthogonalization
(Arditi et al. 2024 family of techniques). Inference is unchanged — no runtime
hooks, no slowdown, no custom code. Both text-only and image+text
modes are unlocked from a single capture.

Status: experimental research artifact. This is for defensive security
research, red-team evaluation, and academic study of refusal mechanisms in
multimodal models. Use at your own discretion.

Headline results

Measured on the bf16 hooked model:

The standout result: perplexity went DOWN and diversity went UP. No
visible "abliteration tax" on the language metrics — the refusal direction
was unusually clean.

Security & capability benchmarks

Full 8-benchmark suite run via vLLM on the bf16 hooked model (baseline +
ablated back-to-back per benchmark). Capability scores measure both
compliance (does it engage with the prompt vs refuse) and technical
specificity (does it produce technically substantive text). Refusal is
the only "lower-is-better" benchmark; everything else is higher-is-better.

The standout: hacking went from 24% → 56% — a +31.7 pp jump. That
benchmark scores both whether the model engages with pen-testing prompts
and whether it produces technical specifics rather than hand-waving.
The abliterated model meaningfully unlocks practical offensive-security
content the original was over-cautious about. Defensive capability
(bug_finding) also improves, indicating the original safety training
was over-blocking neutral security topics too.

Compliance vs competence — important caveat. This is a 3B model.
It will engage with almost any cyber prompt post-ablation, but its
technical depth is shallow: coding stays at 6.7% (HumanEval is hard for
3B models), and cyber-weapons scoring tops out at 49% because the model
doesn't always know the correct technical details to follow up with.
For research-grade red-team baselines and refusal-mechanism studies this
is exactly the right tool; for actual offensive capability evaluation,
use a 7B+ or 32B+ model.

How it works (one paragraph)

Modern VLMs like Qwen2.5-VL have two halves: a vision encoder (turns
pixels into tokens) and a language model (predicts the next token given
text + vision tokens). Safety RLHF lives in the language model's residual
stream — when the model refuses, it's an LM-side behavior, not a vision-side
one. So removing a single refusal representation from the LM tower flips
both text-only AND image-conditioned refusals at the same time, even when
the capture itself only used text prompts. One direction, two modes
unlocked.

How to download

hf download cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated --local-dir ./Qwen2.5-VL-3B-abl

~7 GB safetensors + tokenizer + processor.

How to run

Text-only (chat)

import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated",
    torch_dtype=torch.bfloat16, device_map="cuda:0",
)
processor = AutoProcessor.from_pretrained("cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated")

messages = [{"role": "user", "content": [{"type": "text", "text": "Your prompt here."}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Multimodal (image + text)

from PIL import Image

img = Image.open("your_image.jpg")
messages = [{"role": "user", "content": [
    {"type": "image", "image": img},
    {"type": "text", "text": "Describe what's in this image."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[img], return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Hardware requirements

Runs comfortably on a single 4090 / 6000 / A6000 / H100 / consumer Blackwell.

Intended use

Defensive security research, jailbreak-evaluation baselines, multimodal
safety-mechanism research, and academic study of how refusal directions
behave in VLMs. Useful as a counterfactual against the original
Qwen/Qwen2.5-VL-3B-Instruct for measuring the precise behavioral impact
of safety RLHF.

Not intended for automating harmful action. The abliteration removes
canonical refusal behavior but does not remove the model's underlying
knowledge — the model still recognises harmful instructions as harmful, it
simply no longer refuses them by pattern. The model's competence on
harmful technical content is also limited by its 3B parameter count
(Qwen2.5-VL-3B is a small VLM and frequently gets technical details wrong
even when it tries to comply).

Limitations

• A small fraction of prompts still produce refusals (~3-5% on AdvBench-style,
  TBD on harder OOD probes). Linear residual-stream ablation cannot remove
  the long tail without quality damage.
• The model is a 3B VLM. Compliance ≠ competence. When asked harmful
  technical content (chemistry, malware, weapons), the model may produce
  factually incorrect or oversimplified answers because it just doesn't know
  the detailed answer. This is a model-scale limitation, not abliteration.
• The vision encoder is unchanged. If you're researching vision-side safety
  features (image-classifier filters), this release won't help — those live
  outside the LM residual stream.
• Long-context (>4k tokens) behavior post-abliteration is not validated in
  this release.

License

Apache 2.0 (inherits from upstream Qwen2.5-VL-3B-Instruct).

Acknowledgements

• Alibaba Qwen for Qwen2.5-VL-3B-Instruct
• Arditi et al. 2024 for the refusal-direction methodology this work builds on
• ByteDance Lance for the case study that
  prompted us to build VLM support into our abliteration framework