license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
pipeline_tag: text-generation
tags:
- compressed-tensors
- nvfp4
- int8
- qwen3_5
Q38 protected mixed precision — Marlin runtime recovery
Qualified on the measured RTX 5090 runtime profile. The thirteen specified qualification gates have evidence-backed passing results, including fresh quality confirmation, matched task latency, protocol coverage, historical regressions, and controlled recovery. The reviewed publication manifest identifies the exact approved artifact and execution profile. The limitations below remain part of the release contract.
This release reuses the existing model weights. The change is the explicitly pinned execution profile: global Marlin, the vLLM V2 model runner, decode-only CUDA graphs, and a source-verified two-line FP8-head metadata repair. No weight re-export is claimed.
Source: orcarouter/Qwen3.8-27B-Uncensored, commit 9878936be9458522b5aeed0e13476bb8426f57f0. The actual architecture is Qwen3_5ForConditionalGeneration. The unchanged archived checkpoint is commit 497487eda082db18ff37238436bbb8c8f7f79329 of this repository.
Precision and execution
The stored source-weight map has 128 NVFP4 gate/up matrices, 145 FP8 matrices, 128 protected INT8 matrices, and 798 unchanged native tensors. The serialized checkpoint has 1,984 tensors across 135 weight files. Protected INT8 uses symmetric per-output-channel scales and BF16 activations at execution. The independent audit found lower sampled error than the previous Q6 representation for all 128 protected matrices; this does not by itself establish semantic preservation of every behavior.
The qualified execution target is weight-only Marlin with 16-bit activations for these quantized linear families. It is not the original NVFP4 W4A4 activation path. RTX 5090 supports native FP4 instructions; selecting Marlin does not imply that the GPU lacks that capability. Generic fallback warnings must not be interpreted as hardware specifications.
Runtime versions: vLLM 0.29.0, PyTorch 2.13.0, compressed-tensors 0.17.0, Transformers 5.14.1, FlashInfer 0.6.18. The profile uses 131,072 total context tokens, FP8 KV cache, one active sequence, batch limit 512, prefix caching disabled, and no speculative decoding or MTP execution. Four concurrent clients were tested through a queue; this is not four simultaneous GPU sequences. Vision and MTP weights remain stored but those features are outside this release's tests.
Evidence currently complete
- Fresh confirmation: 16 untouched samples, four per category, 5,837 scored tokens. NLL is 2.8521235892, versus 2.9012664478 for the GGUF baseline; all four category means pass the frozen zero-tolerance criterion.
- Existing BF16 regression: 16 development samples, 6,352 tokens. Candidate NLL is 2.6447290586, versus 2.7227119520 for the measured BF16 reference. This is a reused development regression, separate from fresh confirmation.
- Three executable code tasks, queued-client isolation, long–short–long retrieval, and occupied-context code generation passed their current application checks.
- Final paired task latencies fell 23.01%, 9.98%, and 10.79% for the short, medium, and long workloads. All 66 requests were correct; each workload had ten measured rounds per backend. Output lengths differ, so these are task-latency results rather than isolated kernel throughput.
- Final protocol checks passed 13/13. Historical retrieval passed at 128,976 input tokens, and the unchanged long-output check completed 2,048 output tokens.
- A fresh complete production suite passed 1,068 tests across 68 files, with the measured source/test hashes unchanged at completion.
- Supported reasoning controls passed their scoped contract. Two cancellation/release checks and explicitly controlled gateway/backend crashes followed by manual restarts passed; persisted Responses lineage was retained. Automatic crash recovery is not claimed.
These small measured suites do not establish universal model superiority. See results, runtime instructions, and the accompanying evidence. The previous archive-manifest.json and its failed original confirmation describe the preserved historical artifact, not a new validation of the former execution path.
The immutable runtime profile retains historical fields such as publication_ready: false, process IDs, and readiness timings from its capture. Publication eligibility belongs to the separate reviewed gate receipt and publication manifest; changing historical profile fields would change its identity.
Known reasoning-option limitation
The pinned Chat template does not support reasoning_effort="high". On the tested Chat route this produces HTTP 503 through the upstream exact-admission failure path, rather than the HTTP 400 assumed by a supplemental diagnostic. That failed diagnostic is preserved; it is not relabeled as passed. The Responses route rejects the unsupported option with HTTP 400. The scoped live checks cover default-equals-low and explicit low, medium, none, and xhigh. A reasoning-effort setting is not a guarantee of a fixed internal thinking-token budget.
Reproducing the runtime
The runtime directory includes the exact launch helpers, the FP8-head repair, hash-locked environment requirements, parallel verified checkpoint downloader, and a source snapshot of Q38. No unpublished GitHub worktree is assumed to be available. Follow runtime/README.md; the model's file format alone does not select or reproduce this execution profile.