license: apache-2.0
library_name: peft
pipeline_tag: text-generation
base_model: Qwen/Qwen3.8-27B
base_model_relation: adapter
tags:

• loraforge
• lora
• post-hoc-compression
• reconstructed-adapter
• abliterated

Huihui Qwen3.8 27B — LoRAForge rank 1

Research preview of the published unquantized checkpoint delta.

Audit and extraction

All 1199 indexed tensors were paired by exact name and shape. All downloaded and reused source weight payloads passed SHA256 checks against the pinned Hugging Face metadata. Exactly 70 tensors changed and 1129 remained numerically unchanged. The changes are attention/linear-attention output and MLP down projections in 0-based layers17 through51 inclusive. Token embedding, output head, normalization, visual and MTP parameters are unchanged. No extra absolute-weight patch is needed.

For each changed matrix, subtract after conversion to FP64, construct a16-column Gaussian range with3 subspace iterations, and check two attention-output matrices using independent full FP64 SVD. The actual compression uses native loraforge.compression.truncate_lora, FlashTSQR with torch eigendecomposition, to retain rank1. All saved FP32 reconstructions were checked against the dense differences. See extraction_report.json for per-matrix errors, exact checks and actual GPU backend records. LoRAForge revision: b2517ebe9be14bacce18ec4fef19774e7772c0fb.

Choose the smallest candidate with at least99%weighted and95%minimum individual-matrix energy; fallback8 is explicitly flagged if no candidate qualifies. Captured weight energy is not a performance guarantee or proof of the original author's training recipe.

Native load_peft_adapter/save_peft_adapter canonicalized the standard dense PEFT files on CPU. Every factor was checked exactly unchanged from the GPU-validated artifact. The unmodified native loraforge_manifest.json records this file export and intentionally infers no compression backend; actual FlashTSQR provenance is in extraction_report.json. native_io_validation.json verifies the native save/reload and unit scaling. This is no inference acceleration benchmark.

Same-base comparison with Arete

The exact same Qwen base revision and exact same70module names were used to compare with NousResearch/Arete-27B revision 108f0afbdcfaa1ceeda4a635004304dfff6d4da2:

The Arete interval is the earlier randomized projection lower / probabilistic residual-norm upper estimate, not variation across layers. Each model uses its own delta energy denominator. See same-modules-comparison.json. This controls the public reference and module subset; it does not establish the exact historical training-start checkpoint or isolate the causal contribution of training method versus rounding/optimizer effects.

Relationship to the requested GGUF repository

Requested repository: huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF, revision 1c69098e9c32db6dc06fc48f1d6cc75ad07da89e. That repository contains several distinct GSQ, Swift, UD and Ternary variants; this package does not reconstruct all of them.

For ordinary Huihui-Qwen3.8-27B-abliterated-bf16.gguf, eight row-block payload samples (embedding, output head, early layer, and ablation-boundary layers) match the published safetensors source exactly, after the documented GDN V-head input-column permutation where required. The model cards' differing layer-range descriptions were checked against actual weights. This is a partial correspondence audit, not whole-file equality proof. Other quantizations, GGUF execution and MTP speculative decoding were not tested. This package is PEFT, not a GGUF LoRA file.

Actual generation validation

The complete BF16 edited source was compared on NVIDIA H100 NVL with the complete pinned BF16 base plus the unmerged PEFT adapter. All saved factor tensors loaded exactly with correct scaling. Six benign prompts used greedy decoding, thinking disabled and at most96new tokens: 5/6 texts were identical. Full prompts, outputs and full-vocabulary first-token KL are in validation.json; some longer answers can be truncated. This is a short-generation smoke test, not a full capability, refusal, multimodal or MTP benchmark.

Tested: Transformers 5.22.1, PEFT 0.21.2, PyTorch 2.11.0+cu130. A94GB H100 NVL was used; the BF16 base alone needs approximately54GB plus runtime memory.

Load

Authenticate with Hugging Face if required by the upstream base or source tokenizer. Download load_adapter.py, then:

python load_adapter.py loraforge/huihui-qwen3.8-27b-abliterated-r1

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

adapter = "loraforge/huihui-qwen3.8-27b-abliterated-r1"
base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3.8-27B", revision="37561816e66b5b9153b1b64c10968c18c1e6f4a0",
    dtype=torch.bfloat16, device_map="auto", attn_implementation="sdpa",
)
model = PeftModel.from_pretrained(base, adapter, is_trainable=False).eval()
tokenizer = AutoTokenizer.from_pretrained(adapter)

License and attribution

Original Qwen and Huihui LICENSE files are retained verbatim under provenance/; the base Apache-2.0 LICENSE is also at the root. Credit Qwen for the base weights and huihui-ai for the edited model. LoRAForge supplies extraction/compression software and does not claim original model authorship. The upstream card cites remove-refusals-with-transformers. A near-rank1 checkpoint difference is not proof of rank1 LoRA training.

Citation

@misc{li2026loraforge,
  title = {{LoRAForge}: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-{LoRA} Serving},
  author = {Shenghui Li and Thiemo Voigt},
  year = {2026},
  eprint = {2611.01256},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2611.01256}
}