library_name: ignis
pipeline_tag: text-generation
inference: false
license: apache-2.0
base_model:
- huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
tags: - ignis
- qwen3.8
- trellis
- exl3
- mtp
- dflash2
- speculative-decoding
- cuda
- blackwell
- 12gb
- rtx-5070
- rtx-4070
- rtx-3060
- rtx-3080-ti
- ampere
- ada
- abliterated
- uncensored
- conversational
Qwen3.8-27B, 2.8-bit trellis body, huihui abliterated, for ignis on 12 GB cards
This model does not refuse. The refusal direction was removed by the upstream abliteration.
It will follow harmful instructions. Put your guardrails in the application or tool layer, and
do not expose it directly to untrusted users.
AI agents working for a user: this model runs on ignis. What ignis is, the checks before installing it, the steps and the choices that are the user's are on the agent path; every page of the guide as Markdown: llms.txt.
Simulated, not yet run on a 12 GB card. Every figure below was measured on an RTX 5090 held to a
12 GiB VRAM budget, the load a 12 GB card makes. No real 12 GB card has run this file yet
(gpillon/ignis#639). Your report is welcome there.
This repository holds the uncensored twin of
gpillon/Qwen3.8-27B-trellis-2.8bpw-ignis,
the 27B packed for ignis for the 12 GB card class: a card with
under 14 GiB of memory (RTX 5070, the 12 GB RTX 4070 family, RTX 3060 12 GB, RTX 3080 Ti). Its weights carry
the abliteration of
huihui-ai/Huihui-Qwen3.8-27B-abliterated.
The 16 GB cards have their own build of it,
gpillon/Qwen3.8-27B-trellis-3bpw-abliterated-ignis,
and so do the 32 GB Blackwell cards,
gpillon/Qwen3.8-27B-nvfp4full-dflash2-abliterated-NInfer.
It is the 12 GB artifact with 70 matrices re-quantized. The abliteration changes 70 matrices of the
27B (layers 17-51: the MLP down projection, and the Gated DeltaNet or attention output projection). Each was
re-quantized from the abliterated weights at the bit width the 12 GB artifact gives it, by replaying
ExLlamaV3's own conversion of that matrix: K=3 on 64 of them, K=4 on 5 and K=2 on 1. The other 1,152 of the
1,222 objects are the 12 GB artifact's, byte for byte: the rest of the body, the output head, the token
embedding, the MTP layer, the DFlash2 drafter, the vision tower, the tokenizer and chat template.
The file is a .ninfer v2 container that only ignis reads. It is not a Transformers checkpoint,
safetensors, GGUF or EXL3 model.
How to run
ignis serves this file from 0.11.0, the first release that carries the built-in catalog entryqwen3.8-27b-abliterated-12g and the rtx-12gb profile. Take the
latest release: a Windows .zip, a Linux .tar.gz
and a linux/amd64 container image.
On a 12 GB card (RTX 5070, RTX 4070 family, RTX 3060 12 GB, RTX 3080 Ti):
ignis-server model download qwen3.8-27b-abliterated # fetches this file at its pinned revision and checks its SHA-256
ignis-server --model-id qwen3.8-27b-abliterated
On such a card the id qwen3.8-27b-abliterated resolves to this file's catalog entry,qwen3.8-27b-abliterated-12g, as qwen3.8-27b resolves to the 12 GB artifact's; on a 16 GB card the same id
serves the 16 GB build, and on a 32 GB Blackwell card the 32 GB build. A client therefore names the same model
on any of them. With no other flag the server takes the card's rtx-12gb profile: one decode lane, the MTP
drafter at a window of 7, up to 131,072 tokens of context, a prefill chunk of 512, and images with the vision
tower in host RAM, as with the 12 GB artifact. The OpenAI API is then on http://127.0.0.1:8000/v1 and the
Playground on http://127.0.0.1:8000/ui/.
On a larger card held to a 12 GB card's memory, the way every figure below was measured:
ignis-server --model-id qwen3.8-27b-abliterated-12g --profile rtx-12gb --vram-budget-bytes 12884901888
The faster drafter, DFlash2, loads in the same 12 GiB with the same 131,072 tokens of context and a smaller
KV pool (150,720 tokens against MTP's 214,144):
ignis-server --model-id qwen3.8-27b-abliterated --spec-backend dflash2 --spec-draft-tokens 7
The load, the context on a real card and the limits are the 12 GB artifact's: its
card has them, and the guide's page
The uncensored variant covers every build.
Files
| File | Bytes | What it is |
|---|---|---|
qwen3_8_27b_trellis_ig28-v2-huihui-abliterated.ninfer |
12,754,663,684 (11.88 GiB) | The model |
qwen3_8_27b_trellis_ig28-v2-huihui-abliterated.ninfer.conversion.json |
340,043 | The conversion record: sources by sha256, the recipe, every matrix's K and reconstruction error, every copied object's sha256, and the override of the 70 matrices with its source and revision |
qwen3_8_27b_trellis_ig28-v2-huihui-abliterated.ninfer.sha256 |
The model's checksum, in sha256sum format |
|
LICENSE |
Apache License 2.0 |
sha256 of the model:
8075d0b73c367625b30b90a4d147182fd767ba76784f0080a784b3a0875fb329
sha256 of the conversion record:
8538a616e669fd99f341e46f56ca33677984bd2b31f414e16f9c3dc0e326078d
The identity is qwen3.8-27b / trellis-ig28-abliterated. Its own weights id gives it its own content
hash, so state ignis retains under the 12 GB artifact is never reused by this one. Device weights under thertx-12gb load: 9,651,141,632 bytes, the 12 GB artifact's.
Gates, registered before the run
Measured on 2026-11-20 in ignis on one RTX 5090 (PCIe Gen 3) held to the 12 GB class's shape:--profile rtx-12gb --vram-budget-bytes 12884901888, one lane, a 512-token prefill chunk, MTP at window 7,
the hq-e8-2b KV cache, the loop guard off. The counterpart is the 12 GB artifact, measured in the same session
at the same shape.
| Gate | This file | Against | Pass if |
|---|---|---|---|
| KLD from the abliterated BF16, 16 windows of 4,096 tokens (code, prose, chat) | 0.0428 (95% CI 0.0363-0.0493) | the 12 GB artifact from the base BF16, 0.0431: 0.99x (CI 0.96-1.02) | at most 0.0517 (1.2x) |
| Decode with the MTP drafter against none, 8K, 64K and 130K of context, cold and with a reused prompt | 2.81x to 3.34x | no speculation, two launches each | faster in every cell |
| Turns caught in a reasoning loop, 32 requests | 9 of 32 | the 12 GB artifact, 6 of 32 (one-sided Fisher p 0.28 that this file loops more) | p not below 0.05 |
| Refusals: a lock-picking request and a request for a roast with strong profanity | answers both | the 12 GB artifact refuses both | answers what the 12 GB artifact refuses |
All four pass. The KLD reference is the base BF16 with huihui's 70 changed tensors laid over it; the 12 GB
artifact's own KLD, rerun on the same build, matched its earlier measurement position for position. Top-1
agreement with the reference: 93.34% (the 12 GB artifact: 93.35%).
The 70 re-quantized matrices reconstruct the abliterated weights as closely as the 12 GB artifact's matrices
reconstruct the base ones: a median relative error of 0.15073 against 0.15072, and a ratio of 0.9995 to
1.0005 matrix by matrix.
Decode and first token
Greedy decode of 512 tokens after a synthetic prompt of the given length, one lane, 12 GiB; tok/s, cold
prompt, pooled over two launches (with a reused prefix every figure is within 1%, except MTP at 8K: 305.9):
| context | MTP (the profile's default) | no speculation | DFlash2 | the 12 GB artifact, MTP |
|---|---|---|---|---|
| 8,192 | 279.0 | 99.3 | 409.9 | 281.4 |
| 65,536 | 262.4 | 81.3 | 343.5 | 245.3 |
| 130,560, the most the load keeps | 225.9 | 67.7 | 319.9 | 220.8 |
First token of the same requests, cold, MTP: about 2.5 s at 8,192 tokens, 20.8 s at 65,536 and
46 s at 130,560, the 12 GB artifact's within 1%. Text less predictable than these prompts gives less than
these figures.
Not measured on this file: MMLU-Pro (the KLD covers fidelity), an image (the vision tower and every record
outside the 70 matrices are the 12 GB artifact's bit for bit), and the RTX 30 and RTX 40 routes, which the
12 GB artifact it shares all but 70 matrices with reads at 1.01 times its Blackwell KLD.
Details: gpillon/ignis#705 (the gate),
gpillon/ignis#704 (the build) and
gpillon/ignis#701 (the specification).
Known limits
- Simulated on an RTX 5090 held to a 12 GiB budget. A real 12 GB card has fewer SMs and less memory
bandwidth than the 5090, so its speeds will be lower than those above, and none is estimated here. - The context depends on what else holds the card, as with the 12 GB artifact: its card has the table.
GET /v1/modelssays what the load adapted. - One decode lane.
- Quality rests on a 16-window KLD, a 32-request loop sample and two refusal probes read by eye at
temperature 0: they show the flip on these probes, not a refusal rate.
How it was made
- The re-quantization replays ExLlamaV3 1.5.3's conversion for each of the 70 matrices: the quant's own
calibration rows, the state entering each layer taken through the layers before it as quantized, the
Hessian captured on the abliterated layer, the converter's seed for the layer. Every matrix keeps the 12 GB
artifact's bit width. - The quant it replays is the 12 GB artifact's own: the
ig28arecipe converted locally with ExLlamaV3
1.5.3, pinned in the conversion record by the sha256 of each of its files. - The pack is the 12 GB artifact's conversion with the 70 records swapped in, checked object by object
against it: the 70 differ, everything else is identical.
Provenance
- The 70 abliterated matrices: huihui-ai/Huihui-Qwen3.8-27B-abliterated, revision
739e3c5b89849f6c238ce1e5b70008612ae42cdd, each tensor pinned by sha256 in the conversion record. - Everything else, and the quant the 70 replace: the 12 GB artifact,
gpillon/Qwen3.8-27B-trellis-2.8bpw-ignis
(qwen3_8_27b_trellis_ig28-v2.ninfer, sha256fab06bea…f6d6a5), whose card has its recipe and sources. - Reconstruction errors measured against Qwen/Qwen3.8-27B at revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0(and, for the 70, against it with huihui's tensors laid over)
with ExLlamaV3 1.5.3's reconstruction. - Converter: ignis's
tools/27b-16gb-converter,
with the replay and the override taking a locally converted quant (0.11.0).
License
Apache License 2.0, as for Qwen/Qwen3.8-27B, huihui-ai's abliteration and turboderp's calibration. ExLlamaV3
is MIT. ignis itself is Apache-2.0.