pretty_name: NInfer RTX 5090 Benchmark Evidence
language:

• en
  tags:
• benchmark
• inference
• ninfer
• rtx-5090
• qwen
• nvfp4
• kv-cache
• speculative-decoding
  configs:
• config_name: summary
  data_files:
  • split: test
    path: data/measurements.csv

NInfer on one RTX 5090 — recorded benchmark evidence

Historical results from 19–20 September 2026, published by dima0000. This is a collection of benchmark evidence, not a model checkpoint or a live inference service. It preserves successful measurements, failed checks and incomplete experiments.

Hardware: one NVIDIA RTX 5090 with 32 GB VRAM per trial. Models: regular and uncensored Qwen3.8-27B with NVFP4 weights. Engine: NInfer commit 9e163eee4b8acec21ab0ac765107b6a3f287b217, CUDA 13.1; uncensored runs include the recorded frontend compatibility patch. Host CPU/RAM assignments and some serving configurations differed between rentals.

Models used

The uncensored source repository is gated; users may need to sign in and accept its access conditions. The final NVFP4-head experiment also planned to use that same uncensored revision, but no weight download, conversion or GPU inference ran in that attempt. This dataset contains results and provenance, not model weights or a downloadable converted uncensored artifact.

Start here

Machine-readable selected measurements are a derived summary, not additional experiments or raw statistical samples. Filter by metric, unit, workload and configuration before comparing rows. Full evidence is authoritative.

Performance summary

Single-request decoding: uncensored model

The matched median ratio was 2.106×. Requests were cold; model artifact, prompt order, seeds, temperature zero and output budgets were controlled. Draft counters confirmed the backend choice. This small comparison does not establish batch/long-context speedups or output-quality equivalence.

Short-prompt concurrency: regular model

About 1,005 input tokens plus 512 output tokens per request; FP8 KV and MTP3.

These rates include cold prefill and completion of the entire wave; they are not decode-only rates. Sixteen simultaneous submissions to eight lanes completed at 724.6 aggregate tokens/s, with the excess requests queued—not sixteen active lanes. An eight-request workload with 27,844-token prompts achieved 299.1 aggregate output tokens/s including prefill.

Two uncensored four-request cold waves achieved 468.7 and 440.6 aggregate tokens/s, using different, much shorter prompts. An initial different four-way workload achieved 391.32 tokens/s. These are not controlled regular-versus-uncensored comparisons.

KV capacity and long-context behavior

Four-lane shared GPU KV capacity increased from 276,480 FP8 tokens to 495,360 NVFP4 tokens: +79.17%. The per-request ceiling remained 262,144. A matched large wave took 66.61 s with NVFP4 versus 63.35 s with FP8, about 5.1% longer. The demonstrated benefit was capacity, not throughput.

The uncensored retry submitted prompts of 261,557 / 131,644 / 65,839 / 32,958 tokens, generating 512 tokens each. All four marker checks passed. The whole wave took 171.44 s, about 11.95 aggregate output tokens/s including the huge cold prefills.

Allocated KV occupancy reached 494,080 / 495,360 slots (99.74%), with four running requests in the same sample. One was still prefilling and three were decode-ready. No sampled interval proved that all four entire prompts were simultaneously fully prefilled and resident. The earlier wave peaked at only two running requests. Both are retained; changes to submission order and prefix-cache settings were not isolated experimentally.

A separate regular-model 260,621-token prompt passed retrieval with a first-token latency of 101.08 s. Cold long-context prefill dominates latency. Shared capacity is not capacity per request.

RAM-backed history restoration

Uncensored follow-up: four independent histories, deliberately restricted 131,072-token GPU KV capacity, 16 GiB host KV budget, and a non-retained approximately 124K-token eviction request.

Post-eviction continuations restored 1,771,978,752 bytes (1.6503 GiB) of main-plus-draft KV from host RAM to GPU and reused 98.97% of their input. All memory/tag checks passed in the clarified follow-up. The first scenario also restored KV but failed one peer-tag extraction; its overall failure is retained.

Phase output lengths differed, so cold/continuation timing ratios are not matched speedups. Host RAM preserves inactive histories; it does not enlarge the active GPU pool. Logical history correctness and physical cache reuse are distinct, and previous_response_id does not guarantee a cache hit. IDs and caches are process-local.

Latest head experiment: no GPU performance result

The implementation proposed W4A16 NVFP4 output and shortlist heads, not native W4A4 activation acceleration. Its 510.94 MiB calculated combined head-payload reduction is not measured VRAM savings. No GPU compilation, real-checkpoint conversion, numerical validation, throughput, draft-acceptance or quality comparison completed. Its failure was orchestration-related, not evidence for or against the proposed optimization.

Complete data and integrity

• Original three-trial archive: 39,299,987 bytes, containing all 3,545 retained source files (222,796,366 source bytes) plus package metadata. SHA-256: 7c42b88d70e7ded4e2f06b2759901ccd2f629bf434f3522b184dc684650a478a.
• Original file manifest, checksums, exclusions, archive verifier, and original index are unchanged.
• Fourth-attempt archive: 184,663 bytes. SHA-256: 74d6ab12061c3c1eec4f69bc01967268a93a2beaac609721d2fe1ce2ae9b8198. Its directory also contains the full retained addition, manifests, source changes, failed logs and transfer evidence.
• browse/trials/ duplicates the smaller result/evidence trees for web browsing. Full engine source snapshots and original checkpoint metadata remain in the complete baseline archive rather than being duplicated here.
• Publication manifest and publication checksums cover this upload snapshot. Screening record documents the checks.

The collection contains captured JSON/JSONL, generated model replies, GPU CSV, reports, scripts/patches, source snapshots, metadata, logs, failure evidence and dated provider/billing snapshots. “All results” means all retained available evidence—not uncaptured/truncated output or unrun tests. Some evidence is duplicated between raw archives and browsable copies; do not count duplicates as independent measurements.

Safety, limitations and reuse

• No model weights, private keys, account tokens or signed download manifests are included. Automated screening also checks nested archives; it is not an absolute guarantee against every conceivable sensitive datum.
• Publication of technical metadata was authorized. Files retain historical local paths, cloud resource IDs, endpoint addresses and public host-key information for provenance.
• All four trial pods were verified absent at their recorded cleanup checks. Old RUNNING fields, URLs and launch scripts are historical, not live-resource instructions. Do not connect to reassigned endpoints or execute deployment/guard scripts without fresh authorization and ownership verification.
• The collector is a historical script tied to original Linux paths. Do not rerun it on this downloaded snapshot. The verifier only reads files and is suitable for checking the archive.
• No broad quality, vision, head-quantization-fidelity, exhaustive Unicode or endurance conclusion is established. Retrieval/tag tests are bounded checks, not comprehensive evaluations. Billing values are dated snapshots; some were incomplete estimates.
• See LICENSE-NOTES.md. Publication does not relicense included third-party code or checkpoint metadata.

To verify a downloaded repository snapshot:

sha256sum --check PUBLICATION-SHA256SUMS
sha256sum --check SHA256SUMS
python3 verify-archive.py