license: apache-2.0
tags:
- gguf
- text-generation
- magicquant
- conversational
- mtp
base_model: - llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved
MagicQuant Hybrids (v2.0) - Qwen3.6-27B Uncensored (By llmfan46)
MagicQuant is a benchmark driven GGUF hybrid discovery and validation system focused on finding real, practical GGUF quants specific to each architecture.
Whether it's a pure baseline model built by llama.cpp, learned tensor configurations from Unsloth, or a custom built MagicQuant hybrid, the model table below shows quants that have won dominance checks, survived collapse spaces, and/or were found to be nonlinearly better. Instead of dumping every quant type possible, MagicQuant tests, validates, and brutally murders anything deemed unworthy.
MagicQuant Info & Wiki
MagicQuant is a project intended to become open source. Currently the full methodology is documented at the MagicQuant Wiki.
That wiki will eventually have the code uploaded and be renamed to just "MagicQuant" for the repo instead of "MagicQuant-Wiki". The code is still too much in the early prototype stage. It's beginning to mature, but it requires a heavy hand throughout the process.
Once I'm confident in the code, believe the methodology and protocols are mature enough that I won't be changing it weekly and bricking every setup after every update. I'm excited to share not just the entire methodology, but the entire code base to reproduce MagicQuant :)
Support MagicQuant
I’m a solo developer working full time for myself to achieve my dream. I build open source code on the side. If you like any of my work, buying me a coffee is always appreciated. Otherwise, I hope you enjoy, maybe give me a star or something. Or just send me good vibes. Either way, thank you!
Click here to see ways to support - BTC, Paypal, GitHub sponsors.
Clone Notice
This repository did not run through the full MagicQuant evolution/search pipeline. It is a clone of the final survivor tensor configurations from magiccodingman/Qwen3.6-27B-MagicQuant-GGUF, rebuilt and benchmarked locally for this model.
The archived MagicQuant JSON files in magicquant-manifest/ are copied from the source release for durability. The clone benchmark JSON and the table below are from this clone run, so those metrics reflect the rebuilt outputs in this repository.
Final survivors
| Name | Provider | KLD | Size (GB) | Download |
|---|---|---|---|---|
| MQ-Q6_K_1 | MagicQuant | 0.003838 | 27.70 | Link |
| MQ-Q6_K_3 | MagicQuant | 0.005859 | 24.10 | Link |
| MQ-Q5_K_S_1 | MagicQuant | 0.006975 | 22.34 | Link |
| MQ-Q5_K_S_2 | MagicQuant | 0.007902 | 21.31 | Link |
| LM-Q5_K_S | llama.cpp | 0.015510 | 19.16 | Link |
| MQ-IQ4_NL_1 | MagicQuant | 0.017155 | 18.03 | Link |
| LM-IQ4_NL | llama.cpp | 0.025073 | 16.29 | Link |
| LM-IQ4_XS | llama.cpp | 0.028951 | 15.57 | Link |
| MQ-IQ3_M_1 | MagicQuant | 0.044483 | 14.94 | Link |
| LM-IQ3_S | llama.cpp | 0.065563 | 12.91 | Link |
| LM-IQ3_XXS | llama.cpp | 0.098076 | 11.67 | Link |
| LM-IQ2_M | llama.cpp | 0.165493 | 10.49 | Link |
| LM-IQ2_S | llama.cpp | 0.212110 | 9.85 | Link |
| LM-IQ2_XXS | llama.cpp | 0.302324 | 8.92 | Link |
- This model architecture had unusual anomaly detection occurrence. MagicQuant pipeline utilized this anomaly to achieve unusually better quants than normally achievable. Please read the wiki to understand what a quant anomaly is and how it's utilized.
- MQ-Q6_K_2 removed even though referenced in manifest files. The Q6_K_2 strictly lost to Q6_K_1 meaning the cloning process didn't translate perfectly for that specific quantization pattern. Which is not a big deal but is why I removed it. No reason to have a quant that's just worse than another both in KLD and still larger in size.
MTP Support Notes
Please note that Q8_0 was used for the MTP tensors because as of the time this was added, llama.cpp imatrix and MTP tensor support isn't working fully. Therefore the manifest files won't perfectly represent the new tensors added. Not that it's a huge deal tbh.
This is a more unique situation due to MTP tensors stripped previously but re-added later.
Provider credits
- llama.cpp — Baseline quantization formats and llama.cpp tooling.
Warning - Is MagicQuant Better? (hint: how you frame the question matters)
External/custom baselines are normalized into MagicQuant's controlled comparison flow. MagicQuant rebuilds a learned baseline under native-source / MagicQuant-controlled conditions, including its own imatrix handling, so hybrids or external baselines (like Unsloth) can be judged on a more equal footing. That does not mean MagicQuant proved the original upstream artifact or upstream imatrix was worse. These comparisons exist for internal hybrid-search consistency and equal playing field comparisons, not as a universal judgment of the original creator's exact release artifact.
Easier to digest explanation:
MagicQuant compares and benchmarks the models quant to tensor configurations, but not the original artifact. And there's different reasons MagicQuant chooses to lift up a winning quant, not all winners are purely "better". It depends heavily on a variety of factors. Though choices are always documented in the repo under the manifest folder. You can always view what and why decisions were made by the automated system.
So, MagicQuant can confidently tell you, "under the same quantization to tensor configurations and identical imatrix, with this benchmark, I deemed this a winner".
Re-Uploading External Provider Baselines
By default, if an external provider like Unsloth is deemed the winner, the repo should generally link directly to the original provider instead of re-hosting the quant. External GGUFs are normally only re-uploaded when a specific winning variant does not already exist (e.g. Heretic models or similar).
Release metadata
- Final survivor metrics — full file names, KLD, PPL, PPL delta %, byte sizes, download targets, and replacement lineage. PPL delta % is measured against the native/reference PPL when available; negative is better and larger positive values are worse.
- Hybrid tensor map — tensor-group assignments and effective-state details for MagicQuant hybrid GGUFs.
- Clone tensor configs — exact per-GGUF tensor quantization maps for reproducing this final output list in repository clone mode.
- Isolation samples — isolated base/group probe samples with KLD, PPL, PPL delta %, and size truth.
- Bad trade details — structured bad-trade pruning decisions from the isolation optimizer.
- Clone benchmark summary — fresh benchmark results from this clone run.
- Replacement details — structured details for baselines or anchors removed from the final download table, including reason codes, KLD deltas, PPL delta %, and size deltas.
Replacement reason codes
STRICT_DOMINANCE— the winner was no larger and had lower real KLD than the removed anchor.NEAR_BASELINE_PREMIUM— the winner used only the configured near-baseline size premium and beat the real linear KLD trade line.INTERIOR_DISCOVERY— the winner was selected as a useful interior point inside a size/KLD gap between anchors.SPACING_COLLAPSE— two candidates were too close in practical output space, so the stronger one was kept.FINAL_DOMINANCE— a later validated survivor dominated this artifact in final real benchmark comparison.
Underlined names in the table replaced or ultimately inherited the replacement of another artifact. Hover the name for the short replacement summary, or inspect magicquant-manifest/magicquant.replacements.json for exact KLD/PPL/size deltas.