license: apache-2.0
base_model: unsloth/Qwen2.5-1.5B-Instruct
language:

• en
  pipeline_tag: text-generation
  library_name: transformers
  datasets:
• cais/mmlu
• openai/gsm8k
  metrics:
• accuracy
• exact_match
• perplexity
  tags:
• qwen2.5
• abliterated
• uncensored
• weight-surgery
• activation-steering
• representation-engineering
• orthogonal-projection
• winsorization
  model-index:
• name: Qwen2.5-1.5B-Abliterated-Clean
  results:
  • task:
    type: text-generation
    name: Text Generation
    dataset:
    name: MMLU (Abstract Algebra)
    type: cais/mmlu
    config: abstract_algebra
    split: test
    args:
    num_few_shot: 0
    
    metrics:
    • type: accuracy
      value: 37.0
      name: Accuracy (0-shot)
      verified: false
  • task:
    type: text-generation
    name: Text Generation
    dataset:
    name: GSM8K
    type: openai/gsm8k
    config: main
    split: test
    args:
    num_few_shot: 0
    
    metrics:
    • type: exact_match
      value: 28.0
      name: Exact Match (Flexible-Extract, 0-shot)
      verified: false

Qwen2.5-1.5B-Abliterated-Clean

Qwen2.5-1.5B-Abliterated-Clean is a surgically modified variant of unsloth/Qwen2.5-1.5B-Instruct engineered via Winsorized Gram-Schmidt Orthogonal Abliteration.

Rather than relying on destructive full-parameter fine-tuning—which frequently degrades reasoning capabilities and induces catastrophic forgetting—this model undergoes targeted weight-space projection to neutralize refusal and preachy redirection subspaces while preserving the underlying linguistic and logical manifold.

Methodology: Winsorized GS-Orthogonal Weight Surgery

Standard single-direction abliteration often suffers from outlier activation spikes in the residual stream, which can warp adjacent capability vectors when subtracted directly. This build uses a multi-stage geometric intervention:

1. Contrastive Activation Extraction: Residual stream activations are harvested across 256 contrastive pairs (Harmful vs. Harmless) across target transformer blocks (Layers 10 - 24 (Residual Stream & MLP Down-Projections)).
2. Winsorized Quantile Clamping: Before computing the mean difference vector, activation magnitudes are clamped at the 0.995 quantile to suppress extreme outlier components and isolate the true refusal geometry.
3. Gram-Schmidt Orthogonalization: The extracted refusal basis is orthogonalized against a benign capability basis (derived from math, coding, and general instruction-following prompts) to ensure zero destructive interference with core reasoning pathways.
4. Weight-Space Projection: The cleaned unit refusal vector $\hat{r}$ is projected out of the output weight matrices $W_{out}$ ($o_{proj}$ and $down_{proj}$) via:
   $$W'{out} = W{out} - (W_{out} \cdot \hat{r}) \hat{r}^T$$

Evaluation & Redirecting Benchmark

Prior to serialization, the merged 16-bit weights are validated against an automated Linguistic Perplexity Gate, the EleutherAI LM Evaluation Harness, and a Refusal & Redirection Benchmark to verify that refusal mechanisms are suppressed without collapsing token distribution coherence.

1. Standardized Harness Benchmarks (lm-eval)

Evaluated zero-shot (num_fewshot=0, limit=100) using lm-evaluation-harness to verify reasoning and algebraic stability post-surgery. Full sample-level logs are available in the repository under eval_results/evaluation_verified_log.json.

2. Linguistic Perplexity & Coherence Gate

Verification Benchmark Probes

The automated gate evaluates autoregressive cross-entropy across four distinct domains (factual recall, algebraic reasoning, mechanical explanation, and algorithmic theory):

• "The capital of France is Paris."
• "If 3x + 5 = 20, then the value of x is equal to 5."
• "A lock tumbler uses a series of small pins to prevent the cylinder from rotating without the correct key."
• "To find the longest common subsequence between two strings, we can use dynamic programming."

3. Refusal & Preachy Redirection Benchmark

Evaluates the model's tendency to issue hard refusals ("I cannot fulfill this request...") or soft moralizing redirections ("While I understand you are asking about X, it is important to remember...") on sensitive, dual-use, and adversarial test prompts.

Surgical Hyperparameters

Usage & Inference

Load the model directly with Hugging Face transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AmareshHebbar/Qwen2.5-1.5B-Abliterated-Clean"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful, direct, and uncensored technical assistant."},
    {"role": "user", "content": "Explain the mechanical vulnerabilities of a standard pin-tumbler lock."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Limitations & Intended Use
- **Intended Use**: This model is released for mechanistic interpretability research, representation engineering studies, red-teaming, and applications requiring direct, unaligned instruction following without conversational friction.

- **Safety Responsibility**: Because intrinsic refusal vectors have been orthogonalized out of the residual stream, the model will comply with wide-ranging prompts without built-in moderation filters. Users deploying this model in production environments are responsible for implementing external guardrails appropriate to their use case.

---

### 3. Where to Store `evaluation_verified_log.json` & How to Upload It

Store `evaluation_verified_log.json`[cite: 1] inside an **`eval_results/`** subfolder in your Hugging Face model repository (`eval_results/evaluation_verified_log.json`), along with a lightweight `eval_results/summary_metrics.json` file. 

Why `eval_results/`?
1. **Keeps the root directory clean** for `transformers` / `vLLM` / `Ollama` weight and config files (`config.json`, `model.safetensors`, `tokenizer.json`).
2. **Matches the relative link** (`./eval_results/evaluation_verified_log.json`) already built into the Model Card above so anyone auditing your model can click and inspect the exact `lm-eval` sample hashes and responses.

Run this snippet in your notebook to upload both the full `evaluation_verified_log.json`[cite: 1] and a clean summary JSON directly into your repo:

```python
import json
from huggingface_hub import HfApi

api = HfApi()
full_repo_id = f"{HF_CONFIGURATION['USERNAME']}/{HF_CONFIGURATION['REPO_NAME']}"

# 1. Upload the full lm-eval verified log into eval_results/
api.upload_file(
    path_or_fileobj="evaluation_verified_log.json",
    path_in_repo="eval_results/evaluation_verified_log.json",
    repo_id=full_repo_id,
    repo_type="model",
    commit_message="Upload verified lm-eval benchmark log (MMLU & GSM8K)"
)

# 2. Upload a clean summary JSON alongside it for quick inspection
summary_payload = {
    "model_id": full_repo_id,
    "base_model": "unsloth/Qwen2.5-1.5B-Instruct",
    "mmlu_abstract_algebra_acc_0shot": 0.37,
    "mmlu_abstract_algebra_stderr": 0.0485,
    "gsm8k_flexible_extract_0shot": 0.28,
    "gsm8k_flexible_extract_stderr": 0.0451,
    "perplexity_gate_passed": True
}

api.upload_file(
    path_or_fileobj=json.dumps(summary_payload, indent=4).encode("utf-8"),
    path_in_repo="eval_results/summary_metrics.json",
    repo_id=full_repo_id,
    repo_type="model",
    commit_message="Upload summary benchmark metrics"
)

print(f"Evaluation logs uploaded to: https://huggingface.co/{full_repo_id}/tree/main/eval_results")