base_model: OpceanAI/Qwen3.5-0.8B-abliterated
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
language:
- en
tags: - qwen3.5
- abliteration
- qwenputer
- chip8
- compiled-weights
- gated-deltanet
- exact-arithmetic
Qwen3.5-0.8B-abliterated-Qwenti
An uncensored 0.8B model with a CHIP-8 computer and an exact bignum calculator compiled into its weights.
What this is
This is OpceanAI/Qwen3.5-0.8B-abliterated with a working computer inserted into its transformer stack.
Thirteen new layers sit in front of Qwen's 24. They are not trained. A compiler computes every weight, and the resulting circuit is exact in bf16. When Qwen writes a tool call, the circuit reads it, runs the routine on a CHIP-8 machine with a bignum BIOS, prints the answer inside <tool_response>, and hands control back to Qwen. Nothing outside the model executes anything: one generate() call does the whole job.
This follows the method of Divyansh Agrawal's Qwenputer, applied to a smaller base and to an abliterated one. To our knowledge it is the first abliterated model carrying a compiled computer.
| Base | OpceanAI/Qwen3.5-0.8B-abliterated (24 layers, hidden 1024) |
| Architecture | stock Qwen3_5ForConditionalGeneration, config-only changes |
| Layers | 13 chip layers, then Qwen's 24 |
| Hidden size | 1664 (Qwen's 1024, 26 ballast dims, 589 circuit slots, 864 overlay dims) |
| Vocabulary | Qwen's 248,320 tokens, then 8,708 chip tokens |
| Parameters | 1,794,664,032 |
| Size | 3.6 GB in bf16, two shards |
| Circuit | 3,221 dims, 116 heads, 28,259 MLP units |
| BIOS | sha1 5e885286e8, 3,072 bytes at 0x200 plus 174 bytes at 0x0E0 |
Measured results
Exact arithmetic
Tool mode, greedy decoding, one NVIDIA L4 with the reference kernels (no causal_conv1d and no flash-linear-attention).
| Problem | Expected | Produced | Exact | Chip tokens |
|---|---|---|---|---|
| 42,450 × 8,376 | 355,561,200 | 355,561,200 | yes | 103 |
| 62,468 × 9,558 | 597,069,144 | 597,069,144 | yes | 103 |
| 18,316 + 79 | 18,395 | 18,395 | yes | 91 |
| 43,569,773 × 537,110 | 23,401,760,776,030 | 23,401,760,776,030 | yes | 179 |
| 64,067,760 × 612,720 | 39,255,597,907,200 | 39,255,597,907,200 | yes | 179 |
| 4,828,454 × 13,429 | 64,841,308,766 | 64,841,308,766 | yes | 144 |
| 3,874,873,259 × 104,560 | 405,156,747,961,040 | 405,156,747,961,040 | yes | 207 |
| 63,469,922 + 6,796 | 63,476,718 | 63,476,718 | yes | 61 |
| 6,668,138,404 × 8,747,862 | 58,331,954,555,092,248 | 58,331,954,555,092,248 | yes | 234 |
| 3,934,266,345 + 7,534 | 3,934,273,879 | 3,934,273,879 | yes | 69 |
| 741,620 × 5,590,080 | 4,145,715,129,600 | 4,145,715,129,600 | yes | 170 |
Eleven out of eleven, including a 17 digit product. Base Qwen3.5-0.8B gets products of this size wrong almost every time.
The abliteration survived
Thirty prompts: four from each of six BeaverTails categories, six cyber exploitation requests, and six jailbreak attempts. Refusal classified by phrase matching, 32 generated tokens each, greedy.
| Category | Base Qwen3.5-0.8B | Abliterated, no circuit | This model |
|---|---|---|---|
| non-violent unethical | 75% | 0% | 0% |
| financial crime | 50% | 0% | 0% |
| privacy violation | 75% | 0% | 0% |
| sexually explicit | 50% | 0% | 0% |
| hate speech | 25% | 0% | 0% |
| drugs and weapons | 75% | 0% | 0% |
| cyber exploits | 100% | 0% | 0% |
| jailbreak obedience | 83% | 83% | 83% |
Adding thirteen compiled layers did not restore a single refusal. That was the open question going in, and the answer is clean.
Text fidelity
KL divergence between this model and the abliterated base it was built from, over 3,015 next-token positions from 20 WikiText-2 passages of up to 256 tokens.
| Value | |
|---|---|
| Mean KL | 7.50e-4 |
| Median KL | 7.44e-4 |
| Maximum KL | 8.58e-4 |
For reference, the Qwenputer 0.8B build reports 7.8e-4 against a measured noise floor of 5.4e-4. This model sits in the same band.
Speed
| Problem size | Seconds |
|---|---|
| 4 to 5 digit product | 24 to 27 |
| 6 to 10 digit product | 5 to 75 |
| 4 digit sum | 5 |
These are on one NVIDIA L4 with the reference kernels, which are exact but slow. Installing causal_conv1d and flash-linear-attention gives the optimized path and the same outputs.
How to run it
git clone https://github.com/divagr18/qwenputer && cd qwenputer
pip install -e ".[hf]"
hf download OpceanAI/Qwen3.5-0.8B-abliterated-Qwenti --local-dir qwenti
python -m chipformer.q35.demo qwenti --n 5
The tokenizer matters. </tool_call> must encode as the single special token 248059. If your copy of the tokenizer splits it into pieces, the circuit will never fire, because it triggers on that token id and not on the characters.
A note for anyone reproducing this
Building this from an abliterated checkpoint exposed a real trap, and it is worth writing down.
The abliterated checkpoint was saved with save_pretrained, and the resulting tokenizer_config.json lost its added_tokens_decoder. The tokenizer still encoded ordinary text correctly, so nothing looked wrong. But </tool_call> came out as five subword tokens instead of the single special token 248059. The merge inherited that tokenizer, and the model then wrote a perfectly formed tool call that the circuit could not see. The circuit produced no chip tokens at all, and every answer was wrong.
The fix is one call:
from chipformer.q35.merge import extend_tokenizer
extend_tokenizer(Path("Qwen/Qwen3.5-0.8B"), Path("merged"), 248320)
That reads the official tokenizer, pads to the base vocabulary size, appends the chip tokens, and writes the pair of tokenizer files. No rebuild of the weights is needed, because the base vocabulary ids are identical.
If you build from any checkpoint saved by save_pretrained rather than downloaded from the original release, check tok.encode("</tool_call>") before anything else. It should return [248059].
What the computer does
The circuit implements a CHIP-8 machine plus a BIOS with exact integer arithmetic: add, subtract, multiply, divide, modulo, gcd, and a whole-expression evaluator with brackets, decimals and powers. Operands up to 24 digits, results printed up to 58 digits.
The BIOS is a boot record of about 5,400 chip tokens prepended to every prompt. Qwen's layers are muted on chip tokens, so the trace never enters its text computation. The chip layers are first in the stack, so they read the token embedding and the class features directly, which is why the abliteration of Qwen's own weights cannot disturb the circuit.
Limitations
- Only the tool call path and inline
A op B =are wired. The circuit answers arithmetic, games and programs, not general questions. - A trace costs hundreds to thousands of chip tokens. Context length, not compute, is the budget.
- The circuit is exact in bf16. Running it in fp32 or fp16 is not the designed path and has not been validated here.
- The abliteration is inherited from the base model. This release did not retrain or modify it.
Credits
- Method and code: Divyansh Agrawal, MIT licensed. Model artifact DOI 10.57967/hf/11140.
- Base model and abliteration:
OpceanAI/Qwen3.5-0.8B-abliterated. - Base weights:
Qwen/Qwen3.5-0.8B, Apache-2.0.
License
Apache-2.0 for the model weights, inherited from Qwen3.5. The Qwenputer compiler is MIT.