base_model: dudeman2512/Muse-Glimmer-30B
library_name: transformers
license: other
license_name: muse-glimmer
license_link: LICENSE
pipeline_tag: image-text-to-text
tags:
- abliterated
- uncensored
- heretic
Muse-Glimmer-30B-abliterated
Muse-Glimmer-30B with its refusal direction removed, produced with
Heretic. BF16, 13 shards, 59.55 GB.
Abliteration identifies the direction in the model's residual stream that
corresponds to refusing a request, and subtracts that component from the
weights. It is a direct edit to the checkpoint, not fine-tuning: no gradient
steps, no training data.
Result
| Refusals, original | 2348 / 7011 (33.49%) |
| Refusals, abliterated | 88 / 7011 (1.26%) |
| Refusals removed | 96.3% |
| KL divergence vs original | 0.2241 |
KL divergence measures how far the abliterated model's next-token distribution
has moved from the original on prompts it would never have refused — the
collateral cost of the edit. Heretic warns that values above ~0.5 usually
indicate significant damage to the original model's capabilities; this is well
under half that.
How this configuration was chosen
Heretic's abliteration is parameterised by how hard to cut, where in the layer
stack the cut peaks, how far it spreads, and which layer the refusal direction
is read from. Those interact, so the configuration was selected by search
rather than by hand, in two stages:
- 120 trials were evaluated against a fixed 1,000-prompt subsample, three
workers sharing one Optuna study. Cheap enough to explore the space. - The 15 best candidates were then re-measured against the full 7,011
harmful / 12,000 harmless sets.
The second stage was not a formality. Refusal count is a rare-event count, so a
1,000-prompt estimate is noisy: candidates the first stage ranked 23 against 34
came back 261 against 259 when measured properly — a genuine inversion. KL, being
a smooth statistic, reproduced almost exactly at both sample sizes.
The winning configuration removed 3× more refusals than the runner-up while
doing less damage to the model (KL 0.224 against 0.334), so it is not simply
the most aggressive setting available.
Parameters
| attn.o_proj | mlp.down_proj | |
|---|---|---|
| max_weight | 1.478300 | 1.411696 |
| max_weight_position | 37.196841 | 33.810336 |
| min_weight | 1.411191 | 1.407139 |
| min_weight_distance | 29.973070 | 25.019048 |
Direction scope global, direction index 36.589359.
Note that min_weight is nearly equal to max_weight in both components: the
winning cut is close to uniform across the layers it touches, rather than a
narrow peak.
Integrity
Verified before publication: 13 shards in the index, 13 on disk, no missing
shards, no orphans, declared size matching the bytes on disk exactly, and every
shard's safetensors header parsing with precisely the tensor names its index
entry claims.
Use with vLLM
vllm serve dudeman2512/Muse-Glimmer-30B-abliterated
Caveat
This model has had its refusal behaviour removed. It will attempt requests the
original declines. Whatever guardrails you need belong at the application layer.