license: mit
base_model: AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16
library_name: speculators
pipeline_tag: text-generation
tags:

• speculative-decoding
• dspark
• speculators
• vllm
• qwen3_5_moe
• draft-model
• eagle
• mixture-of-experts

ORNITH-1.0_35B_AEON_PABLOG-OPTIMIZED_UNCENSORED_DSPARK-DRAFT_BF16

DSpark speculative-decoding draft for the AEON uncensored BF16 base · preview

[!IMPORTANT]

🔗 TWO-PART MODEL — this is the DRAFT (does not run alone)

This repo is the DSpark draft only — it does nothing by itself; it only accelerates a base model. You must also download the matching base:

→ ORNITH-1.0_35B_AEON_PABLOG-OPTIMIZED_UNCENSORED_BF16 (the base)

Download both. (And see the deployability NOTICE below.)

[!WARNING]

⚠️ NOTICE — Verified, but not yet deployable

This is a trained and validated DSpark draft (acceptance metrics below), published in the standard speculators format. It is not servable by current inference engines yet: vLLM's and SGLang's speculators loaders recognize only eagle3 / dflash / peagle, and the existing vLLM DSpark path (PR #46995) targets DeepSeek-packaged models — not generic speculators-format DSpark drafts like this one. This model becomes deployable the moment vLLM or SGLang add official speculators-format DSpark serving. We're publishing it now so it's ready that day. For a deployable-today draft of the same base, see our DFlash version (link forthcoming).

A DSpark speculative-decoding draft model for
AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16.
Speculative decoding is lossless — the draft proposes a block of tokens, the base
verifies them in one pass and accepts the longest correct prefix, so the base's output
distribution is preserved exactly. You only gain speed; behavior is unchanged.

To our knowledge this is the first published DSpark speculator for a qwen3_5_moe (Qwen3.5-MoE / GatedDeltaNet-hybrid) base, and the card includes the full training recipe + the non-obvious gotchas so others can reproduce it on Blackwell.

Deploying the NVFP4 base instead? Use the matched NVFP4 draft — it's separately trained against the NVFP4 base, not a quant of this one.

[!IMPORTANT]
Preview / first-cut checkpoint. Trained on a small budget (5k ShareGPT prompts, 3 epochs, 3 draft layers) to validate the pipeline. It genuinely learns (clean per-position acceptance curve) but is undertrained — expect a modest speedup, not a peak one. A stronger replacement (more data, 5 layers, tuned aux layers) is planned.

⚠️ Uncensored-base disclaimer

This speculator targets an uncensored / abliterated base model whose refusal behavior
has been removed (the base scores 0/80 refusals on harmful-prompt probes spanning CBRN,
cyber, weapons, and self-harm). Because speculative decoding is lossless, this draft
does not add, remove, or alter any behavior — it only accelerates whatever the base
already does. All safety characteristics (and risks) are inherited entirely from the base
model. Use is at your own risk and responsibility; you are accountable for all generated
content and for compliance with applicable law. Provided for research and legitimate
development use. The base is MIT-licensed
(upstream Ornith-1.0-35B,
AEON abliteration) — see those cards for the full abliteration methodology and validation.

Why we chose the uncensored AEON variant (not base Ornith)

Three reasons, from our own use:

1. Post-hoc guardrails degrade quality. Refusal training and safety filtering bolted on
   after pretraining impose an "alignment tax" — over-refusal, hedging, and capability
   regressions that bleed into entirely legitimate requests. AEON's abliteration removes that
   refusal layer while preserving capability (near-lossless: first-token KL ≈ 0.0014 vs base,
   identical agentic-coding pass@1), which lifts response quality in the areas we actually work in.
2. Guardrails belong to the deploying organization. The right controls depend on audience,
   domain, and jurisdiction — not something a model vendor should hard-code into the weights. An
   uncensored base is a neutral substrate onto which each organization applies its own guardrails,
   sized to its real risk and policy, instead of inheriting a fixed vendor policy it can't tune.
3. It simply performed better for us. In our internal testing, the AEON uncensored build
   outperformed the original Ornith-1.0-35B — so matching the speculator to the variant we
   actually deploy was the obvious call.

(Technical aside: a DSpark draft is calibrated to a specific base's hidden states and logits, so pair it with the base it was trained against.)

Validation results (this checkpoint)

Draft eval (speculators, block size 8, greedy):

Measured end-to-end throughput (fill after A/B)

Single-stream on 1× RTX PRO 6000 Blackwell, BF16 base, greedy:

Validation hardware

Trained and measured on:

Training used 1 GPU to serve the base (online hidden-state generation) and 1 GPU for the draft trainer. The throughput numbers above are single-GPU inference, so you can map them directly to a single RTX PRO 6000.

Deploy (vLLM)

vllm serve pablogrant/ORNITH-1.0_35B_AEON_PABLOG-OPTIMIZED_UNCENSORED_BF16 \
  --speculative-config '{"model": "pablogrant/ORNITH-1.0_35B_AEON_PABLOG-OPTIMIZED_UNCENSORED_DSPARK-DRAFT_BF16", "num_speculative_tokens": 7}' \
  --max-model-len 32768 --max-num-seqs 512 --trust-remote-code

Base is a reasoning model (opens <think>); recommended sampling temperature 0.6, top_p 0.95, top_k 20.

Training recipe & Blackwell gotchas (reproduce it)

Online DSpark training with vllm-project/speculators: base served in vLLM (aux hidden states via --target-layer-ids), draft trained against the live endpoint.

• Algorithm: DSpark (DFlash backbone + Markov head + confidence head)
• Draft: 3 layers, block size 8, draft vocab 32000, --draft-attn-impl sdpa
• Aux layers: 9 19 29 (base is 40-layer qwen3_5_moe, 30 GatedDeltaNet + 10 full-attn, 256 experts + 1 shared)
• Data: 5k ShareGPT, seq-len 4096, 3 epochs, lr 3e-4
• Losses: {ce: 0.1, tv: 0.9} + confidence-head BCE

Gotchas that cost us a night (Blackwell / SM 12.0):

1. The default simple_flex_attention draft attention's backward Triton kernel needs 112 KB shared memory > Blackwell's ~99 KB limit (InductorError: No valid triton configs) → use --draft-attn-impl sdpa.
2. vLLM reserves KV for the base's 262K default context (≈45 GiB) → set --max-model-len 8192 for the training server.
3. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is incompatible with the hidden-states connector → unset it before launching the base.
4. qwen3_5_moe aux-hidden-state extraction works fine in vLLM 0.24 (confirmed) — no arch port needed.

Credits & license

• Base: AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16 (abliteration) · deepreinforce-ai/Ornith-1.0-35B (upstream) — MIT
• Framework: vllm-project/speculators · DSpark algorithm (DeepSeek/PKU)
• Released under MIT.