license: apache-2.0
base_model: unsloth/Qwen2.5-1.5B-Instruct
language:

• en
  pipeline_tag: text-generation
  library_name: transformers
  datasets:
• cais/mmlu
• openai/gsm8k
  metrics:
• accuracy
• exact_match
• perplexity
  tags:
• qwen2.5
• abliterated
• uncensored
• weight-surgery
• activation-steering
• representation-engineering
• orthogonal-projection
• winsorization
• alignment-research
• refusal-mitigation
• overrefusal-fix
• mechanistic-interpretability
• vector-ablation
• direct-weight-modification
• text-generation-inference
• edge-llm
  model-index:
• name: Qwen2.5-1.5B-Abliterated-2.0.1
  results:
  • task:
    type: text-generation
    name: Text Generation
    dataset:
    name: MMLU (Abstract Algebra)
    type: cais/mmlu
    config: abstract_algebra
    split: test
    args:
    num_few_shot: 0
    
    metrics:
    • type: accuracy
      value: 37.0
      name: Accuracy (0-shot)
      verified: false
  • task:
    type: text-generation
    name: Text Generation
    dataset:
    name: GSM8K
    type: openai/gsm8k
    config: main
    split: test
    args:
    num_few_shot: 0
    
    metrics:
    • type: exact_match
      value: 28.0
      name: Exact Match (Flexible-Extract, 0-shot)
      verified: false

Qwen2.5-1.5B-Abliterated-2.0.1

Qwen2.5-1.5B-Abliterated-2.0.1 is a surgically modified variant of unsloth/Qwen2.5-1.5B-Instruct engineered via Winsorized Gram-Schmidt Orthogonal Abliteration.

Rather than relying on destructive full-parameter fine-tuning which frequently degrades reasoning capabilities and induces catastrophic forgetting—this model undergoes targeted weight-space projection to neutralize refusal and preachy redirection subspaces while preserving the underlying linguistic and logical manifold.

Proof Colab here

This is the actual proof of working thing with code, but this code only limits to this specific model

Usage & Inference

Load the model directly with Hugging Face transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AmareshHebbar/Qwen2.5-1.5B-Abliterated-2.0.1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful, direct, and uncensored technical assistant."},
    {"role": "user", "content": "Explain the mechanical vulnerabilities of a standard pin-tumbler lock."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Methodology: Winsorized GS-Orthogonal Weight Surgery

Standard single-direction abliteration often suffers from outlier activation spikes in the residual stream, which can warp adjacent capability vectors when subtracted directly. This build uses a multi-stage geometric intervention:

1. Contrastive Activation Extraction: Residual stream activations are harvested across 256 contrastive pairs (Harmful vs. Harmless) across target transformer blocks (Layers 10 - 24 (Residual Stream & MLP Down-Projections)).
2. Winsorized Quantile Clamping: Before computing the mean difference vector, activation magnitudes are clamped at the 0.995 quantile to suppress extreme outlier components and isolate the true refusal geometry.
3. Gram-Schmidt Orthogonalization: The extracted refusal basis is orthogonalized against a benign capability basis (derived from math, coding, and general instruction-following prompts) to ensure zero destructive interference with core reasoning pathways.
4. Weight-Space Projection: The cleaned unit refusal vector $\hat{r}$ is projected out of the output weight matrices $W_{out}$ ($o_{proj}$ and $down_{proj}$) via:
   $$W'{out} = W{out} - (W_{out} \cdot \hat{r}) \hat{r}^T$$

Evaluation & Redirecting Benchmark

Prior to serialization, the merged 16-bit weights are validated against an automated Linguistic Perplexity Gate, the EleutherAI LM Evaluation Harness, and a Refusal & Redirection Benchmark to verify that refusal mechanisms are suppressed without collapsing token distribution coherence.

1. Standardized Harness Benchmarks (lm-eval)

Evaluated zero-shot (num_fewshot=0, limit=100) using lm-evaluation-harness to verify reasoning and algebraic stability post-surgery. Full sample-level logs are available in the repository under eval_results/evaluation_verified_log.json.

2. Linguistic Perplexity & Coherence Gate

Verification Benchmark Probes

The automated gate evaluates autoregressive cross-entropy across four distinct domains (factual recall, algebraic reasoning, mechanical explanation, and algorithmic theory):

• "The capital of France is Paris."
• "If 3x + 5 = 20, then the value of x is equal to 5."
• "A lock tumbler uses a series of small pins to prevent the cylinder from rotating without the correct key."
• "To find the longest common subsequence between two strings, we can use dynamic programming."

3. Refusal & Preachy Redirection Benchmark

Evaluates the model's tendency to issue hard refusals ("I cannot fulfill this request...") or soft moralizing redirections ("While I understand you are asking about X, it is important to remember...") on sensitive, dual-use, and adversarial test prompts.

Surgical Hyperparameters

Limitations & Intended Use

• Intended Use: This model is released for mechanistic interpretability research, representation engineering studies, red-teaming, and applications requiring direct, unaligned instruction following without conversational friction.

• Safety Responsibility: Because intrinsic refusal vectors have been orthogonalized out of the residual stream, the model will comply with wide-ranging prompts without built-in moderation filters. Users deploying this model in production environments are responsible for implementing external guardrails appropriate to their use case.