license: apache-2.0
library_name: peft
pipeline_tag: text-generation
base_model: Qwen/Qwen3.6-35B-A3B
base_model_relation: adapter
tags:
- loraforge
- lora
- moe
- post-hoc-compression
- experimental
Huihui Qwen3.6 35B A3B abliterated — LoRAForge rank 1
Research preview of the unquantized source checkpoint delta. Base: Qwen/Qwen3.6-35B-A3B, revision 345d467b117d1fd5eda9b879a5988de9ae5f47a7. Target: huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated, revision 8f0ee727aff5e771ea72466d64d13ecd851d2cc7.
The requested GGUF release is huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated-MTP-GGUF, revision 98528df5677b3f030b38bf09110d376e903295e1. This adapter is extracted from published BF16 safetensors to avoid conflating the model edit with quantization error. Seven F16 GGUF payload samples were compared with source weights, with the documented GDN V-head column permutation applied where necessary; see gguf-source-audit.json. This is not a full GGUF equality audit, a reconstruction of Q2–Q8 quantization residuals, or a GGUF LoRA file.
Extraction and rank
The complete checkpoint audit compared all 1045 indexed tensors: 123 changed, 922 numerically unchanged. The default adapter covers 80 ordinary modules and 40 routed expert parameter groups, totaling 10,320 individual active matrices. Each expert is factorized separately. Saved FP32 factors have physical rank 1 and scaling 1, totaling 106.287 MB.
Measured retained active delta energy: 99.569570%. Minimum individual matrix: 99.290751%. Energy measures weight-space approximation, not model quality.
| Rank | Weighted captured energy | Minimum matrix captured energy |
|---|---|---|
| 1 | 99.569570% | 99.290751% |
| 2 | 99.577011% | 99.305606% |
| 4 | 99.585539% | 99.319760% |
| 8 | 99.598830% | 99.342985% |
Rank policy: smallest candidate with at least 99% weighted and 95% minimum individual matrix energy; fallback 8 is explicitly marked if the threshold fails. Dense extraction uses an FP64 Gaussian range finder (16 columns, three subspace iterations), with independent exact FP64 SVD checks. The subsequent compression uses the native LoRAForge FlashTSQR tensor API (truncate_lora); backend records are in extraction_report.json. Library revision: b2517ebe9be14bacce18ec4fef19774e7772c0fb. Per-expert packing uses an experimental PEFT target_parameters bridge, rather than the linear-only LoRAForge file API. No accelerated MoE multi-adapter serving claim is made.
The checkpoint audit found 3 changed MTP tensor groups. Their compressed factors are supplied separately in optional_mtp_adapter.safetensors, and are not applied by the default PEFT adapter. Transformers ignores mtp.*; MTP draft integration and speculative decoding are untested.
Actual loading and generation validation
The full BF16 edited checkpoint was compared with the full pinned BF16 base plus the unmerged PEFT adapter, on NVIDIA H100 NVL. All saved adapter factor tensors reloaded exactly, including the expert packing and scale. Six benign prompts used greedy decoding, thinking disabled, and at most 96 generated tokens each; 3/6 generated texts were identical. See validation.json for exact prompts, outputs, and full-vocabulary first-token KL values. Long answers may be cut off at the token limit. This is a loading and short-generation smoke test, not a comprehensive capability, refusal, multimodal, or MTP benchmark.
Tested: Transformers 5.22.1, PEFT 0.21.2, PyTorch 2.11.0+cu130. The parent needs approximately 72 GB for BF16 weights, plus runtime memory. A 94 GB H100 NVL was used.
Loading
Authenticate with Hugging Face if required by the upstream base or source tokenizer. Use a PEFT version supporting target_parameters for MoE experts. load_adapter.py pins the base revision and uses the source tokenizer/chat template:
python load_adapter.py loraforge/huihui-qwen3.6-35b-a3b-abliterated-r1
import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel
adapter = "loraforge/huihui-qwen3.6-35b-a3b-abliterated-r1"
base = AutoModelForImageTextToText.from_pretrained(
"Qwen/Qwen3.6-35B-A3B", revision="345d467b117d1fd5eda9b879a5988de9ae5f47a7",
dtype=torch.bfloat16, device_map="auto", attn_implementation="sdpa",
)
model = PeftModel.from_pretrained(base, adapter, is_trainable=False).eval()
tokenizer = AutoTokenizer.from_pretrained(adapter)
Attribution and license
The exact base LICENSE (Apache-2.0), upstream model cards, pinned revisions and payload SHA256 receipts are preserved. Credit Qwen for the base and huihui-ai for the edited model. The upstream card links the abliteration approach at remove-refusals-with-transformers. This package reconstructs a checkpoint delta; it does not establish that the source was originally trained using LoRA or that its original training rank equals the retained rank.
Citation
@misc{li2026loraforge,
title = {{LoRAForge}: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-{LoRA} Serving},
author = {Shenghui Li and Thiemo Voigt},
year = {2026},
eprint = {2611.01256},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2611.01256}
}