library_name: peft
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
tags:
- lora
- qlora
- continued-pretraining
- text-generation
Lamarck-Qwen3.8-27B-abliterated-v2
A LoRA adapter for
huihui-ai/Huihui-Qwen3.8-27B-abliterated,
produced by continued pretraining (QLoRA: 4-bit NF4 base, bf16 compute).
Rank 32, alpha 64, ~233M trainable parameters — double the rank of the
original Ronin48LLC/Lamarck-Qwen3.8-27B-abliterated
(r=16), which remains available unchanged. The adapter does not modify the base
weights.
This version was trained with validation-aware early stopping and
best-checkpoint selection. The released adapter is the best checkpoint
(step 1000, eval loss 1.5875).
The adapter is base-specific — load it on the abliterated base above (the
upstream base has had refusal-related directions ablated), not on the pristineQwen/Qwen3.8-27B.
About the name
Named for Jean-Baptiste Lamarck and the inheritance of acquired characteristics.
The base model is the frozen germline; the adapter is the acquired trait layered
on top during fine-tuning, inherited by anyone who loads base + adapter.
Lamarckian inheritance, shipped as a small file.
Usage
Requires a recent transformers (the base is a Qwen3_5ForConditionalGeneration).
from transformers import Qwen3_5ForConditionalGeneration, AutoTokenizer
from peft import PeftModel
base = Qwen3_5ForConditionalGeneration.from_pretrained(
"huihui-ai/Huihui-Qwen3.8-27B-abliterated", device_map="auto")
model = PeftModel.from_pretrained(base, "Ronin48LLC/Lamarck-Qwen3.8-27B-abliterated-v2")
tok = AutoTokenizer.from_pretrained("huihui-ai/Huihui-Qwen3.8-27B-abliterated")
Metrics
Four-way comparison under an identical harness (4-bit NF4 base, single L40S):
the unedited base, the original r=16 adapter, an r=32 fixed 3000-step run, and
this adapter (r=32 validation-aware, 1000 steps). Perplexity — lower is better.
| Metric | Base | r=16 (v1) | r=32 (3000 steps) | v2 (this adapter) |
|---|---|---|---|---|
| Held-out security perplexity | 6.13 | 5.77 | 4.77 | 4.88 |
| OOD CyberMetric perplexity | 22.25 | 17.73 | 18.70 | 17.76 |
| OOD MMLU-security perplexity | 8.58 | 7.91 | 7.85 | 7.75 |
| OOD SecQA-v2 perplexity | 4.91 | 4.45 | 4.37 | 4.19 |
| OOD math-control perplexity | 6.31 | 6.57 | 6.35 | 6.26 |
| MMLU-security accuracy | 0.820 | 0.815 | 0.820 | 0.815 |
| SecQA-v2 accuracy | 0.98 | 0.99 | 0.98 | 0.99 |
| GSM8K accuracy | 0.907 | 0.940 | 0.927 | —* |
* GSM8K generation for this adapter did not complete (benchmark runtime cap
during this secondary check); the math-control perplexity above already shows
it forgets the least of the four configurations.
Finding: this adapter matches the 3000-step run's in-distribution security
fit while winning every out-of-distribution generalization measure and the
math control, at one-third the training steps. The fixed 3000-step run shows
the classic overfitting signature: extra steps improved in-distribution fit
marginally but degraded generalization across the board. Multiple-choice
accuracy is at ceiling (~0.82 / ~0.98) and cannot separate the adapters;
perplexity is the sensitive measure. These are research measurements, not a
capability claim.
Limitations
A research adapter. Verify outputs; no warranty of fitness for any particular
purpose.