library_name: starlight
pipeline_tag: text-generation
inference: false
license: other
license_name: qwen-community-1.0
license_link: LICENSE
base_model:

• Qwen/Qwen3.8-Flash-Next
• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
  base_model_relation: quantized
  tags:
• starlight
• qwen3.8
• moe
• quantized
• mtp
• abliterated
• amd
• nvidia

Qwen3.8-Flash-Next iq3_s_turbo (abliterated) for Starlight

For Starlight only. This repository holds a model file made for one program, the inference engine Starlight. It is not a Transformers or Safetensors checkpoint, and it is not meant for llama.cpp, Ollama, LM Studio or any other program: other programs are neither supported nor tested.

The abliterated twin of the iq3_s_turbo file of Qwen3.8-Flash-Next, a mixture-of-experts language model, as one 64.78 GiB file for Starlight. One 16 GB graphics card of either maker runs it: the experts that do not fit on the card stay in system RAM and are read as needed. The draft head for multi-token prediction (MTP) is inside the file, and the vocabulary is pruned to 141,312 tokens.

At a glance

What is different from the plain file

The refusal direction is removed in the weights: the 144 matrices that write into the residual stream are edited. Of these, 95 keep the calibrated GSQ-RCO numbers (the 2-bit and 4-bit expert and shared-expert down projections: each column received the smallest code changes that cancel the refusal direction), and 49 are ordinary 4-bit to 8-bit matrices (attention output and state-space output) that were quantised again, the standard way, from the full-precision weights after the edit. Nothing else in the file differs from the plain file. The abliteration is in the weights, so any program that loads the file gets the abliterated model.

Measured by the Starlight project against the plain iq3_s_turbo file:

The last two rows are within the run-to-run spread (about 5 to 12 %); the project's records call them parity.

What is inside

The vocabulary is pruned from 248,320 to 141,312 tokens (141,148 real tokens and 164 padding entries); the 107,172 dropped tokens are removed from the model, and every kept token has the same text as in the original.

Memory and speed, as measured

Test machines: a Ryzen 5 7500F with 62 GiB RAM and NVMe; the RX 9070 XT sat on a PCIe 5.0 x8 link and the RTX 5070 Ti on a PCIe 5.0 x4 link, because a second card was installed (a missed expert is a PCIe copy, so the same RTX 5070 Ti on a x16 link decoded about a third faster in an earlier test). Medians of three runs, 20 October 2026, 64 GB RAM class (38 GiB assigned to Starlight), with this file.

On a 32 GB RAM class machine the RX 9070 XT decoded about 30 tok/s (first token 9.2 s at 8k, 24 s at 32k) and the RTX 5070 Ti about 35 tok/s (8.7 s, 22 s). Two RTX 5060 Ti (tensor split, 21 Oct 2026, i9-12900K, 62 GiB RAM) decoded 96 to 109 tok/s over 2k to 64k of context, first token 2.5 s at 8k and 10.4 s at 32k. A result moves by 5 to 12 % between runs, because the text a model writes sets how many draft tokens are accepted and how often an expert is reused.

Context window: 245,760 tokens on one card of either maker (a 236,286-token prompt was read in 70 s on the RX 9070 XT and in 60 s on the RTX 5060 Ti and answered correctly), and the model's own limit of 262,144 on two RTX 5060 Ti or, with the attention cache overflowing into system RAM, on one card.

Quality

Measured by the Starlight project against the published BF16 reference logits of the base model (jyohukuchan/qwen38-flash-next-bf16-logits): 12 text cases, 16,561 scored positions, Starlight engine on an RTX 5070 Ti, 2 November 2026. Lower KLD (mean KL divergence to BF16) and lower perplexity are better; top-1 is how often the file's most likely next token is the BF16 model's.

What the paired comparison on the same positions supports:

• The IQ3_S reference is clearly the closest to BF16 (KLD 0.065 below iq3_s_turbo, top-1 2.6 points higher), at 13 GiB more.
• iq3_s_turbo and the IQ3_XXS reference are tied (the KLD gap of 0.018 is within noise), and iq3_s_turbo is 6 GiB smaller.
• iq3_xxs_turbo has a KLD 0.034 higher than the IQ3_XXS reference (a real difference); top-1 does not clearly differ, and its perplexity is lower, though not clearly so.
• iq3_xxs_turbo_mini and iq3_xxs_turbo differ by 0.008 KLD, which is noise, and the mini file is 4.65 GiB smaller.

The engine's KL reading is about 2 % below an exact full-vocabulary calculation (checked on one case), so compare the rows with each other rather than with numbers published elsewhere.

The quality table was measured with the plain iq3_s_turbo file (row 2); the abliterated file differs from it only by the edit described above.

About abliterated models

"Abliterated" means that the direction in the model's internal activations that carries refusals has been taken out of the weights. Requests that the original model would decline are answered. It is not a different or smarter model, and it has no content filter of its own: it will not warn, refuse or moderate.

• You are responsible for what you ask it and for what you do with its answers. Follow the law and the licence of the base model.
• Do not offer it to people who have not chosen an unfiltered model, and add your own safeguards where a use needs them.
• Everything the base model's card says about errors, bias and factual reliability still applies; removing refusals does not make answers more correct.

Pictures

The vision encoder is not inside this file. For picture input, put the separate vision file mmproj-Qwen3.8-Flash-Next-BF16.gguf (907,542,944 bytes, MD5 e95e4e6f9aa47c1c30519ba8d288b337) in the same folder as the model file; Starlight finds it by itself and uses it unchanged. It is published in ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF and is not part of this repository. Without it the model answers text only.

How to use

1. Download the file into the models folder of your Starlight installation:

   hf download feverdoingwork/Qwen3.8-Flash-Next-iq3_s_turbo-abliterated-Starlight Qwen3.8-Flash-Next-abliterated-GSQ-RCO-iq3_s_turbo.starlight.gguf --local-dir <models folder>
   

2. Open the Starlight Launcher. It lists the file as Flash (abliterated): it recognises the file by its MD5.

3. Choose your card and press Start. The Launcher starts the Starlight engine program for that card (AMD or NVIDIA) and serves the model on your own machine through an OpenAI-compatible HTTP interface.

Verify the download

echo "1a0a3cd93c269e1c3cf93ed0a0a705392e952ebbd4088651d9d74f484f699b29  Qwen3.8-Flash-Next-abliterated-GSQ-RCO-iq3_s_turbo.starlight.gguf" | sha256sum --check

The same line is in SHA256SUMS. The Starlight Launcher knows the file by its MD5, a5e83c015532bd67cd1d33ac5979a1bd.

Related files

All four are the same base model, packed for Starlight:

• feverdoingwork/Qwen3.8-Flash-Next-iq3_s_turbo-Starlight: 64.78 GiB, the base recipe.
• feverdoingwork/Qwen3.8-Flash-Next-iq3_s_turbo-abliterated-Starlight: the same file, abliterated.
• feverdoingwork/Qwen3.8-Flash-Next-iq3_xxs_turbo-Starlight: 54.68 GiB, trellis-coded experts.
• feverdoingwork/Qwen3.8-Flash-Next-iq3_xxs_turbo_mini-Starlight: 50.03 GiB, trellis-coded experts and a smaller token table.

Limits

• Starlight only (see the notice at the top). The file is not a general-purpose checkpoint.
• Experts that do not fit on the card are read from system RAM. The speeds in this card were measured on 64 GB machines (about 48 GiB or more of RAM available). On a 32 GB machine the one-card setups decoded at 30 to 35 tokens per second, and with less free RAM than that the engine switches to a slower copy path.
• Speeds come from an earlier Starlight engine build (20 and 21 October 2026) on one test machine each. They use Starlight's default cache-aware routing, which biases the expert choice slightly toward experts already on the card; in those tests it made decode roughly 20 to 40 % faster than the plain router.
• Pictures need the separate vision file (see Pictures).
• Quantisation costs quality, as the table above shows. The base model's own limits apply, and like every language model it can state wrong things with confidence.

License and credits

• Licence: Qwen Community License 1.0, the licence of the base model; the full text is in LICENSE. In short, as the licence says: keep its notice with every copy; if a commercial product or service has more than 100 million monthly active users or US$ 20 million in monthly revenue, the model name must be shown in its interface; and a Model-as-a-Service or AI Work Assistant business needs a separate licence from Qwen. Read the licence for the exact terms.
• Base model: Qwen3.8-Flash-Next by the Qwen team.
• Calibrated quantisation: the GSQ-RCO quantisation of ISTA-DASLab (ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), which this file builds on.
• Abliteration, packing, recipe and the Starlight engine: the Starlight project.