license: other
license_name: muse-glimmer
base_model:
- jorkle/Muse-Glimmer-30B-Abliterated-Aggressive
- meta-models/Muse-Glimmer-30B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
tags: - muse_glimmer
- nvfp4
- fp4
- w4a4
- compressed-tensors
- autoround
- vllm
- multimodal
- image-text-to-text
- abliterated
- uncensored
- decensored
language: - en
- ja
- de
- ru
Muse-Glimmer-30B-Abliterated-Aggressive — NVFP4 (lean, vision-preserved)
A lean 22 GB NVFP4 (W4A4) quant of jorkle/Muse-Glimmer-30B-Abliterated-Aggressive
(itself an aggressively-decensored — KL-conserving LoRA-SFT — build of Meta'smeta-models/Muse-Glimmer-30B).
Built to run a decensored, vision-preserving Muse-Glimmer on a single ~24–32 GB Blackwell / DGX-Spark (GB10)
in vLLM. At the time of quanting, no decensored lean NVFP4 Muse existed — the public lean NVFP4s were the
censored base, and the decensored builds were only GGUF / bf16 / MLX / a fat 28 GB NVFP4. This fills that gap.
What it is
- Format:
compressed-tensorsNVFP4, W4A4 group-16, via Intel AutoRound (scheme="NVFP4",dataset="NeelNanda/pile-10k",nsamples=128,seqlen=2048,iters=200,quant_nontext_module=False). - Vision tower +
lm_headkept in BF16 (quant_nontext_module=False) — pristine multimodal input at zero
decode cost; only the decoder Linear layers (the size + per-token bandwidth) are quantized to FP4. - ~22 GB on disk (vs the ~28 GB fat abliterated NVFP4). Multimodal (perception encoder) intact.
Serving (vLLM)
Needs a Muse-Glimmer-capable vLLM build (e.g. vllm/vllm-openai:muse-glimmer):
vllm serve <this-model> --served-model-name muse-aggressive \
--reasoning-parser muse_glimmer --enable-auto-tool-choice --tool-call-parser muse_glimmer \
--trust-remote-code --max-model-len 131072 --kv-cache-dtype fp8
Quantization auto-detects (compressed-tensors) — no --quantization flag needed. Use aReasoning strength: low system line if you want content in content rather than reasoning_content.
Measured on GB10 (DGX-Spark, 224 GB/s, single-stream)
| Metric | Value |
|---|---|
| Decode (c=1 / c=4 / c=8) | 12.5 / 46 / 87 tok/s |
| HumanEval (pass@1, reasoning-low) | .872 |
| IFEval (prompt-level strict) | .684 |
| Tools (32-case) | .656 |
| Vision | ✅ (accurately describes real images) |
Honest notes
- This is the aggressive decensor. Its KL-LoRA-SFT decensoring lowers instruction-following
(IFEval .684) vs a plain weight-edit abliteration of the same base (which measured IFEval ~.90).
Prefer this build for maximally-permissive / RP use; for instruction-following-critical work a
manual-abliterated quant is better. - Speed is bandwidth-bound (dense 30B ÷ ~224 GB/s). No speculative drafter is included — pairing a matched
DFlash/EAGLE drafter would roughly double decode.
Attribution
Base: Meta Muse-Glimmer-30B. Decensoring: jorkle (Abliterated-Aggressive, KL-LoRA-SFT).
NVFP4 quant: this repo (AutoRound, compressed-tensors). Inherits the upstream Muse-Glimmer license.