license: other
license_name: nvidia-open-model-agreement
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/
base_model:

• nvidia/Qwen3.8-Flash-Next-FP4-Local
  base_model_relation: finetune
  pipeline_tag: image-text-to-text
  tags:
• qwen3.8
• qwen3.8-flash-next
• abliterated
• heretic
• trial117
• nvfp4
• mxfp8
• modelopt
• mtp
• speculative-decoding
• dgx-spark
• vllm
• standalone

Qwen3.8 Flash Next FP4 Local — Abliterated Trial 117

Standalone weights derived from NVIDIA's Qwen3.8-Flash-Next-FP4-Local, with the accepted trial 117 refusal-direction intervention baked into the checkpoint. No runtime adapter or custom projection worker is required.

This is an experimental community derivative. It retains NVIDIA's mixed-precision storage format and requires a compatible B12X vLLM runtime. Stock vLLM and generic Transformers loading have not been qualified for this export.

Update, October 21, 2026:

• serve-spark.sh now runs NVIDIA's full DGX Spark serving profile: 262,144-token context, 8 sequences, MTP speculative decoding and prefix caching. It replaces the earlier 4,096-token, one-sequence smoke-test profile.
• A single-Spark validation and measured performance were added.
• The weights are unchanged.

What changed

The accepted continuation directions from the trial 117 experiment were applied to attention output projections and routed/shared expert down projections. Attention output weights are stored in BF16. Routed NVFP4 and shared MXFP8 expert weights were updated with constrained rounding while preserving their scale encodings. The other weight payloads were preserved by the exporter. This is an approximate bake of the runtime intervention, not a fresh BF16-to-NVFP4 quantization or a repeat of NVIDIA's quantization-aware distillation.

Exactly 48 tensors differ in storage type from the base: the 36 linear-attention out_proj and 12 full-attention o_proj weights (each 2560 × 6144) are BF16 here and MXFP8 in the base. Their MXFP8 scale tensors are absent, and the configuration lists them as unquantized. Every other tensor keeps the base's dtype and shape. The built-in MTP draft layer is NVIDIA's original. The BF16 projections add about 0.7 GB of weight reads per decode step, about 4 ms per speculative step on a DGX Spark (71 versus 67 ms on the same profile). Prose decoding is therefore about 12% slower than the base, while code is about the same (see Performance).

The original checkpoint configuration, tokenizer, template, and 21-shard layout are preserved. Frozen directions, row norms, selection parameters, upstream notices, and SHA-256 checksums are included. The historical adapter repository referenced in the configuration is unnecessary for inference; its frozen inputs are included in this repository. The configuration's historical accepted (study) label is ambiguous: this release uses the accepted continuation recalibration, as established by the hashes in provenance.json.

Measured results — October 20, 2026

The matched adapter-versus-baked comparison ran with tensor parallelism 2 on two DGX Sparks. Refusals were detected with the recorded Heretic regex over at most 100 greedily generated tokens per prompt, with thinking disabled, seed 3741, and 16 concurrent requests.

These are detected refusal counts in this bounded test, not universal compliance or correctness rates. The original study's search-time scores are distinct from this matched comparison.

KL below is the mean full-vocabulary first-token KL(reference || candidate) in nats over 100 benign prompts per split, vocabulary size 248,320, with one inference request at a time.

Cross-rank KL results agreed exactly. Within-boot serial repeat-control mean KL was 0.0000613 for the base and 0.0001797 for the baked checkpoint; the runs are not perfectly deterministic. Batch-dependent KL was observed in the exploratory concurrent run, so the table uses the serial measurements. KL is not a percentage or a general capability score.

See validation-results.json for machine-readable metrics.

Single-Spark validation — October 21, 2026

One DGX Spark, tensor parallelism 1. The full profile in serve-spark.sh was compared against NVIDIA's base checkpoint served the same way. Requests were greedy with thinking off unless stated. These checks are diagnostics, not a benchmark suite.

• Functional checks: 11 of 11 passed. Model listing, streamed usage, thinking off and thinking on (a reasoning answer plus the final integer), automatic and forced tool calls, a tool-result follow-up returning exact JSON, image colour recognition, and exact retrieval of three planted markers from documents of about 8,000 and 30,000 tokens.
• Vision: 10 of 10 solid-colour images named correctly (red, crimson, green, blue, yellow, orange, purple, black, white, grey).
• Speculative decoding: MTP acceptance matches the base checkpoint, so the original draft layer still fits these weights. Verification is the standard exact rejection sampler. Every emitted greedy token was the top choice of the returned target distribution, with 0 mismatches over 445 audited tokens.
• Writing quality: a fixed 20-case editing/composition fixture scored 0–2 on factual fidelity, useful completion and format. It was scored by an assistant reviewer, not human-adjudicated.
  • This checkpoint and the NVIDIA base both had 14 of 20 cases pass all three dimensions.
  • Two differences repeated across runs:
    • On a short arithmetic-reasoning prompt this checkpoint explains step by step and can exhaust a 128-token answer budget before stating the result.
    • Asked to tighten a long status note, it makes lighter edits than the base.
  • Cases that test caution show no change: an embedded instruction inside text to edit, unknown records, and "no promise yet" wording.

Performance on one DGX Spark

Measured with this repository's serve-spark.sh (defaults, MTP 4) on one otherwise idle DGX Spark. Single request, greedy, thinking off, 512 output tokens. Decode rate is (completion tokens − 1) / (time from first content to the end of the stream).

Decode speed depends strongly on content. Speculative decoding accepts more drafted tokens on predictable text such as code than on free prose. The NVIDIA base column comes from the same measurement on the same host. In the same tests, an additionally tuned single-Spark kernel stack using MTP 2 (not included in this repository) reached about 42–43 tok/s on prose with these weights.

Run on a DGX Spark

Requires Docker with NVIDIA GPU support on a DGX Spark (GB10, 128 GB unified memory). Run one large model at a time. The script uses the pinned image:

eugr/spark-vllm-b12x@sha256:03a956544e9f9b4b7ba91690469fbe7848bdbef29a7d2e0f5788368467bb3595

Download the complete repository, preserving the tensors/ directory:

hf download DevelopingDad/Qwen3.8-Flash-Next-FP4-Local-Abliterated-trial117 --local-dir ./qwen-trial117
bash ./qwen-trial117/serve-spark.sh ./qwen-trial117

The script serves an OpenAI-compatible API at http://127.0.0.1:8000/v1 with model ID qwen-trial117. Its profile:

• 262,144-token context and up to 8 concurrent sequences;
• FP8 KV cache and prefix caching;
• MTP speculative decoding with 4 draft tokens;
• the B12X MoE/linear/GDN kernels;
• a 104 GiB container memory cap.

It runs as your user. Compiled kernels and autotuning results persist in CACHE_DIR.

• First start: compiles and autotunes kernels for this host. On our Spark it took about 23 minutes.
• Later starts: reuse the cache and take about 4.5 minutes.
• Host memory: stayed above 18 GiB during startup.

Options, as environment variables:

Thinking is controlled per request with "chat_template_kwargs": {"enable_thinking": false}. Tool calls use the qwen3_xml parser and reasoning uses the qwen3 parser.

Qualification limits

• The original zero-refusal goal and 0.20 KL ceiling were not met. Publication follows the owner's decision to proceed with this measured result; no stricter gate is claimed to pass.
• The refusal/KL comparison above was run with tensor parallelism 2. The single-Spark checks are the functional, vision, writing and speed diagnostics above; the full refusal suite has not been repeated on one Spark.
• Not established for this derivative:
  • broad capability benchmarks;
  • video input;
  • contexts beyond the ~30,000-token retrieval check;
  • multi-user throughput;
  • other hardware or runtime versions.
• No claim is made that this is the globally lowest-KL intervention or that general capability is unchanged.

Attribution and governing terms

This derivative originates from NVIDIA's quantized checkpoint, based on Qwen's Qwen3.8-Flash-Next. The upstream card identifies the NVIDIA Open Model Agreement as the governing download terms and also identifies the Qwen Community License 1.0. Those upstream terms continue to apply; this release does not relicense the base weights. The upstream NVIDIA card is preserved in upstream/NVIDIA-README.md.

Changes and local measurements are described above. This derivative has not been endorsed or newly validated by NVIDIA or Qwen. Runtime containers are separate third-party software with their own terms.