library_name: ninfer
pipeline_tag: image-text-to-text
inference: false
license: apache-2.0
base_model:
- huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated
- shisa-ai/Ornith-1.5-35B-A3B-MTP-ONLY
- z-lab/Qwen3.6-35B-A3B-DFlash
base_model_relation: quantized
tags: - ninfer
- ornith
- qwen3.6
- abliterated
- uncensored
- not-for-all-audiences
- quantized
- groupwise-int
- speculative-decoding
- mtp
- dflash
- multimodal
- rtx-3090
Ornith-1.5-35B-A3B Abliterated + Shisa MTP — NInfer
An abliterated Ornith 35B model with an improved speculative-decoding head, packaged for local NInfer inference. It has been run on a single RTX 3090 (24 GB).
The underlying checkpoint comes from Huihui's abliterated Ornith. This build replaces its original MTP head with Shisa's trained head to improve speculative decoding.
| At a glance | |
|---|---|
| Required backend | NInfer, serving the native .ninfer artifact |
| Tested setup | RTX 3090 24 GB with Don-Chad/ninfer-3090 |
| Model download | 21.22 GiB (22,783,246,080 bytes) |
| Text context | Launcher default 160K with watchdog; 120K is an available lower-context option |
| Vision | Included, but off by default; maintainer's 3090 profile is limited to 32K context |
| Other GPUs | May work with compatible upstream NInfer, but not verified for this artifact |
Runtime setup: Use a compatible NInfer executable together with the supplied launcher and watchdog helpers to serve this
.ninfermodel.
Download and run (Windows / RTX 3090)
1. Download the model: Ornith-1.5-35B-A3B-Abliterated-MTP.ninfer.
2. Get the tested backend: Download a compatible ninfer-serve.exe from the NInfer-3090 releases.
3. Put the supplied launcher files together:
Ornith-1.5-35B-A3B-Abliterated-MTP.ninfer
launch-ornith-3090.ps1
ninfer-watchdog.py
watchdog_stream.py
watchdog_probe.py
The launcher requires Windows PowerShell 5.1+ and Python 3 on PATH; its three Python helpers need no third-party packages. The NInfer executable can be elsewhere on disk (provide its path below). The supplied launcher and helpers are part of the release's reference setup.
4. Start the server from PowerShell in the folder containing the launcher:
.\launch-ornith-3090.ps1 -NInferExe 'C:\path\to\ninfer-serve.exe'
For a lower-context text session:
.\launch-ornith-3090.ps1 -NInferExe 'C:\path\to\ninfer-serve.exe' -Context 120000
5. Connect an OpenAI-compatible client, such as OpenCode:
API base URL: http://127.0.0.1:8080/v1
Streaming: enabled (stream: true)
Connect through port 8080 to use the required watchdog. Port 8081 is the direct backend endpoint.
Why the watchdog is required
Testing has been limited to the maintainer's own machine, which needs this watchdog. Other machines may behave differently; whether they need it is unknown. The supplied launcher requires it as part of this reference setup.
The maintainer sometimes encounters infinite/repetitive output when more than roughly 110K tokens of context are actively in use. This refers to the tokens actually in the conversation, rather than the configured context ceiling.
When the watchdog detects the known loop signature, the launcher forcibly stops and restarts the NInfer backend, where it then resumes the job and tries again. Recovery restarts the backend process; the GPU driver stays running.
Why this version?
Ornith's original MTP head was comparatively weak in Shisa's published tests. Replacing it gives NInfer a more useful draft head for speculative decoding — a way to generate tokens faster by accepting predicted tokens in batches.
| MTP head | Overall draft acceptance in Shisa's upstream test |
|---|---|
| Ornith's original head | 32.19% |
| Shisa's KL-distilled head | 60.51% |
Important: These percentages are from Shisa's tests on standard Ornith, not a test of this abliterated .ninfer release. A higher acceptance rate can help speed, but this table is not a measured speedup for this build. Source.
The artifact also contains a DFlash companion, but the validated RTX 3090 path uses MTP with a draft window of four (--draft-tokens 4); DFlash performance is not claimed here.
What performance should I expect?
The maintainer's 4,298 eligible completed requests (October 24–November 11, 2026) provide a real-use reference for a single RTX 3090, not controlled benchmarks:
| Actual prompt size | Median output speed |
|---|---|
| Under 25K tokens | 189.9 tokens/s |
| 75–100K tokens | 143.2 tokens/s |
| 140–160K tokens | 120.1 tokens/s |
These rates include generated reasoning tokens. They are grouped by actual prompt length, not the context setting. Speed varies by request, system load and runtime. They do not establish stability, response quality or performance on another GPU.
Known limitations
- Large-context repetition: Intermittent infinite/repetitive output has been observed above roughly 110K actively used context. This is an operating observation, rather than a measured failure threshold. The bundled launcher uses its watchdog at every context setting to detect the known loop signature and recover the backend.
- 160K is the maintainer's watchdog-protected default, not a guarantee of loop-free operation. Context settings up to 200K are supported by the launcher, but near-full-200K performance and reliability have not been established.
- Vision costs substantial VRAM. On the tested 24 GB RTX 3090, the maintainer could boot vision at no more than 32K context; even that is not a guarantee for every image workload.
- Validation scope: The artifact has loaded and generated with MTP4 on the maintainer's 3090. The updated public launcher has passed PowerShell syntax and offline watchdog/recovery checks, but a live model-load/recovery test of that exact public adaptation was not performed as part of this update.
- Other GPU compatibility and refusal rates remain unverified. No measured refusal rate is claimed for this exact artifact.
Verify your download
The published artifact is 22,783,246,080 bytes (21.22 GiB), with SHA-256:
c3264c17ab26605e864fb18beefbc02929e034a737e8c74c2935093d6a2121f9
Check it against the included SHA256SUMS.txt:
Get-FileHash .\Ornith-1.5-35B-A3B-Abliterated-MTP.ninfer -Algorithm SHA256
Safety, sources and license
Reduced safety filtering: This build uses a checkpoint described by its author as abliterated/uncensored. It may generate sensitive, controversial or inappropriate output. It is not suitable for every audience or unmonitored public-facing deployment; users are responsible for reviewing outputs and complying with applicable law. The replacement MTP head does not restore safety filtering. No refusal-rate measurement has been performed on this exact .ninfer artifact.
This release is distributed under Apache-2.0, with attribution preserved for Ornith-derived MIT-licensed material. See LICENSE, NOTICE and THIRD_PARTY_NOTICES.md.