license: apache-2.0
datasets:
- SargeDev/jev-distill-corpus-v3
- SargeDev/solar-decisions-corpus-v4
base_model: NousResearch/Arete-27B
tags: - abliterated
- uncensored
- qwen3.8
- decision
- nvfp4
Arete-27B-Uncensored-V2-NVFP4-FP8
NVFP4/FP8 quantized build of Arete-27B-Uncensored-V2 — the same weights, ~29 GB instead of ~54 GB, for single-GB10 serving.
Lineage: NousResearch/Arete-27B -> OBLITERATUS aggressive abliteration -> Solar-JEV decision LoRA (solar-training-v3 checkpoint 1946, r32/alpha-64, combined 764k corpus with lm_head target) merged by direct delta arithmetic. See the V2 card for evals, decision-gate numbers, and credit.
Quant recipe (llm-compressor, proven Recipe B from our V3 line)
- attn q/k/v/o + gate/up: NVFP4 W4A4, group 16, FP8 scales
- down_proj: FP8 dynamic per-token
- untouched: lm_head, embed_tokens, mtp (bf16)
- calibration: 384 jev validation judge prompts + 128 ultrachat, MAX_SEQ 2048
Files
- 2 bf16-scale shards + separate
model_mtp.safetensors(~29 GB total) recipe.yaml— the exact quant config, reproducible- GGUF ladder (11 quants): Arete-27B-Uncensored-V2-GGUF
Serve (vLLM, GB10)
--quantization compressed-tensors handles this format. FP8 KV cache recommended.
Credit
Nous Research (Arete base) · Pliny the Liberator (OBLITERATUS) · SargeDev (abliteration run, solar decision merge, quant, evals) · Qwen (base arch).