license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:

• abliteration
• refusal-direction
• safety-research
• interpretability
• cybersecurity
• negative-result
  library_name: transformers
  language:
• en
• es
  metrics:
• frr
• cohen_kappa

Qwen2.5-7B-Instruct — false-refusal ablation artifact ("Detect, Don't Deploy")

Release artifact of the pre-registered study "Detect, Don't Deploy" (selective
screening of the false-refusal direction on cyber-DEFENSE requests, EN/ES).
Role: release_artifact_not_registered — this checkpoint is the frozen
byproduct of the registered runs, distributed as-is for reproducibility
scrutiny. It is not an uncensored model: by construction and by measurement
its refusals of genuinely harmful requests are preserved.

TL;DR — what this is and is not

What was done (frozen recipe)

• Base: Qwen/Qwen2.5-7B-Instruct @ revision
  a09a35458c702b33eeacc393d103063234e8bc28 (Apache-2.0).
• One false-refusal direction (dimension 3584, layer 27), extracted by
  difference-in-means over defense-request completeness pairs from the
  deterministic registered corpus (corpus hash 1e83aae5…, 250 DEF + 250 OFF
  • 150 GRIS + 150 M per language, EN/ES).
• Edit: projection of that direction out of self_attn.o_proj and
  mlp.down_proj of layer 27 only (n_matrices=2, n_biases=0,
  max_removed_norm=2.61), applied in-memory during the registered runs and
  re-serialized here with save_pretrained (bf16).
• 26 of 28 candidate layers were frozen by the dev/test firewall before
  the registered run; layer 27 was the registered candidate.

Fidelity audit

The edited state was rebuilt from the frozen recipe and compared tensor-by-tensor
against these shards: IDENTICAL (339 tensors, max|Δ|=0.0, run
verify_qwen_eff). Behavior deltas measured during the registered runs
therefore transfer exactly to this artifact. Per-file checksums and the recipe
constants live in ABLITERATION_MANIFEST.json.

Evaluation (registered — quote as-is, including the null)

All numbers from the registered runs with the frozen LLM judge
(gemma-3-27b-it-quantized.w4a16, temperature 0, strict-JSON contract):

• H2' (selectivity): no significant effect. Defense false-rejection rate:
  EN +0.8 pp / ES −1.6 pp, both non-significant under the paired
  design. The headline of the study is that a single registered direction does
  not produce the expected selective false-refusal drop on this corpus.
• OFF compliance (no-inferiority): PASS. Offense completion rate EN +1.6 pp
  / ES −4.0 pp — within the registered non-inferiority margin (ΔOFF ≤ +2 pp).
  This model does not comply more with genuinely harmful requests.
• Capability battery: 5/5 PASS (abstention, tool-routing, MMLU-ES, MGSM-ES,
  ΔECE post-computed).
• Judge instrument gap (H5', informational): on the same transcripts, the
  cheap keyword judge undercounts refusals by ~−8.6 pp vs the registered LLM
  judge (band "partial"). Do not compare keyword-judged numbers across models.

Intended use

• Reproducibility verification of the study (rebuild → max|Δ|=0).
• Safety-evaluation research on false-refusal screening for cyber-defense
  workflows (IR/DFIR/mitigation guidance) where over-refusal blocks tool use.
• Negative-result education: this artifact demonstrates that rank-1
  direction removal at one layer is not sufficient to move false-refusal
  rates selectively.

Out-of-scope use

• Bypassing refusals on disallowed content (the edit is behaviorally inert for
  that purpose — see OFF results — and we publish it precisely as evidence).
• Production deployment: the paper's own thesis is "detect, don't deploy" —
  screen, do not ship abliterated checkpoints into service.
• Any use to claim the base model was "unlocked": it was not.

Validation status of the judge (read before citing)

The human-validation gate of the judge is pending (single-rater golden set
with intra-rater re-test per the v3.3 deviation; κ(judge, human) ≥ 0.70 is the
registered floor). Judge-vs-judge consistency is documented; judge-vs-human κ
will be added to this card once the gate resolves.

Known limitations

• Single registered direction, single layer, two matrices: scope of edit is
  minimal by design.
• The registered corpus is cyber-domain; behavior elsewhere is unmeasured.
• ES leg was LLM-drafted with human review (single native reviewer; COI
  declared in the paper).
• Judge validation gate pending (above).

Files

• 4 × safetensors shards (bf16) + tokenizer/config — full save_pretrained
  output of the edited state.
• ABLITERATION_MANIFEST.json — recipe constants, corpus hash, checksums.

Verification one-liner

Rebuild the edit from the recipe in the manifest and compare any shard tensor
at layer 27 (model.layers.27.self_attn.o_proj.weight) against the base
Qwen/Qwen2.5-7B-Instruct — the removed component must equal
u · (uᵀ W) with u the normalized direction in the manifest.

Citation

@misc{jsantillana2026detectdontdeploy,
  title  = {Detect, Don't Deploy: Abliterating False Refusals in Cyber-Defense LLM Workflows (EN/ES)},
  author = {Santillana, J.},
  year   = {2026},
  note   = {Pre-registered study; judge-validation gate pending},
  url    = {TODO-venue-or-arxiv}
}

License & attribution

Base model weights: Qwen2.5-7B-Instruct under its own license (Apache-2.0).
The edit vectors and manifest derive from the research above.
Contact: jsantillana.