license: mit
pipeline_tag: image-text-to-text
tags:
- exl3
- huihui
- abliterated
- vision
| Branch | Weights | Korean TG | Code TG | Actual post-transplant input lengths tested |
|---|---|---|---|---|
| 2.04bpw | 75.97 GiB | 33.7 | 40.0 | 129,996 |
| 2.27bpw | 83.47 GiB | 34.7 | 40.3 | 129,996, 1,046,999 |
| 2.52bpw | 94.08 GiB | 32.4 | 37.1 | 129,996 |
TG is tok/s on short synthetic samples with MTP 3. Every branch is configured for a 1M window; consult the last column for actual post-transplant context tests.
Tested engine: native ExLlamaV3 1.5.4 with the mixed-MLP guard included in each branch, plus TabbyAPI. Read the branch instructions before loading with the pinned stock engine.
Each checkpoint preserves its base EXL3 routed experts, vision, and MTP, replacing Huihui's 18 attention-output B and 18 shared-expert down projections. Those donor tensors are Q8_0 decoded to F16. Branch names denote the base quantization setting; they do not describe a new effective average bit rate.
Select a branch to download weights and read its detailed validation and provenance. main contains this index only.