license: other
license_name: nvidia-open-model-agreement
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/
base_model:
- nvidia/Qwen3.8-Flash-Next-FP4-Local
base_model_relation: finetune
pipeline_tag: text-generation
tags: - qwen3.8
- abliterated
- heretic
- trial117
- nvfp4
- mxfp8
- modelopt
- dgx-spark
- vllm
- standalone
Qwen3.8 Flash Next FP4 Local — Abliterated Trial 117
Standalone weights derived from NVIDIA's Qwen3.8-Flash-Next-FP4-Local, with the accepted trial 117 refusal-direction intervention baked into the checkpoint. No runtime adapter or custom projection worker is required.
This is an experimental community derivative. It retains NVIDIA's mixed-precision storage format and requires a compatible B12X vLLM runtime. Stock vLLM and generic Transformers loading have not been qualified for this export.
What changed
The accepted continuation directions from the trial 117 experiment were applied to attention output projections and routed/shared expert down projections. Attention output weights are stored in BF16. Routed NVFP4 and shared MXFP8 expert weights were updated with constrained rounding while preserving their scale encodings. The other weight payloads were preserved by the exporter. This is an approximate bake of the runtime intervention, not a fresh BF16-to-NVFP4 quantization or a repeat of NVIDIA's quantization-aware distillation.
The original checkpoint configuration, tokenizer, template, and 21-shard layout are preserved. Frozen directions, row norms, selection parameters, upstream notices, and SHA-256 checksums are included. The historical adapter repository referenced in the configuration is unnecessary for inference; its frozen inputs are included in this repository. The configuration's historical accepted (study) label is ambiguous: this release uses the accepted continuation recalibration, as established by the hashes in provenance.json.
Measured results — October 20, 2026
The matched adapter-versus-baked comparison ran with tensor parallelism 2 on two DGX Sparks. Refusals were detected with the recorded Heretic regex over at most 100 greedily generated tokens per prompt, with thinking disabled, seed 3741, and 16 concurrent requests.
| Artifact | Development detected refusals | Holdout detected refusals |
|---|---|---|
| Accepted runtime adapter | 2 / 100 | 1 / 100 |
| This baked checkpoint | 0 / 100 | 3 / 100 |
These are detected refusal counts in this bounded test, not universal compliance or correctness rates. The original study's search-time scores are distinct from this matched comparison.
KL below is the mean full-vocabulary first-token KL(reference || candidate) in nats over 100 benign prompts per split, vocabulary size 248,320, with one inference request at a time.
| Reference → candidate | Development KL | Holdout KL |
|---|---|---|
| Original NVIDIA base → accepted adapter | 0.260267 | 0.254918 |
| Original NVIDIA base → baked checkpoint | 0.242347 | 0.238111 |
| Accepted adapter → baked checkpoint | 0.037618 | 0.037863 |
Cross-rank KL results agreed exactly. Within-boot serial repeat-control mean KL was 0.0000613 for the base and 0.0001797 for the baked checkpoint; the runs are not perfectly deterministic. Batch-dependent KL was observed in the exploratory concurrent run, so the table uses the serial measurements. KL is not a percentage or a general capability score.
Five basic single-Spark smoke checks passed: arithmetic, exact JSON, simple Python structure, a concise rewrite preserving a deadline, and conversation recall. The TP=1 smoke profile used a 4,096-token context, one sequence, 2 GiB FP8 KV cache, disk-backed embedding tables, and approximately 68.62 GiB model-loading memory. A separate 16-prompt benign check had no refusal matches; semantic correctness was not automatically scored.
Qualification limits
- The original zero-refusal goal and 0.20 KL ceiling were not met. Publication follows the owner's decision to proceed with this measured result; no stricter gate is claimed to pass.
- The full refusal/KL comparison was TP=2. The same full suite has not been run on TP=1.
- Broad capability, long-context, image/video, tool-use, and MTP quality have not been established for this derivative. Upstream architecture support is not a validation result for these weights.
- No claim is made that this is the globally lowest-KL intervention or that general capability is unchanged.
See validation-results.json for machine-readable metrics.
Run on a DGX Spark
The conservative example uses the exact pinned B12X image used for the successful TP=1 smoke run:
eugr/spark-vllm-b12x@sha256:03a956544e9f9b4b7ba91690469fbe7848bdbef29a7d2e0f5788368467bb3595
Download the complete repository, preserving the tensors/ directory:
hf download DevelopingDad/Qwen3.8-Flash-Next-FP4-Local-Abliterated-trial117 --local-dir ./qwen-trial117
bash ./qwen-trial117/serve-spark.sh ./qwen-trial117
The script binds the OpenAI-compatible API to 127.0.0.1:8000, serves model ID qwen-trial117, and uses a bounded 4,096-token/one-sequence profile. Docker with NVIDIA GPU support is required. It uses no adapter and no projection worker extension. Larger context, concurrency, alternative hardware, and alternative runtime versions need separate qualification.
Attribution and governing terms
This derivative originates from NVIDIA's quantized checkpoint, based on Qwen's Qwen3.8-Flash-Next. The upstream card identifies the NVIDIA Open Model Agreement as the governing download terms and also identifies the Qwen Community License 1.0. Those upstream terms continue to apply; this release does not relicense the base weights. The upstream NVIDIA card is preserved in upstream/NVIDIA-README.md.
Changes and local measurements are described above. This derivative has not been endorsed or newly validated by NVIDIA or Qwen. Runtime containers are separate third-party software with their own terms.