license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
library_name: mlx-vlm
pipeline_tag: text-generation
base_model:
- orcarouter/Qwen3.8-Flash-Next-Uncensored
tags: - mlx
- dflash2
- speculative-decoding
- apple-silicon
- experimental
Qwen3.8-Flash-Next-Uncensored-mlx-optimized-2-8bit-DFlash2
The optimized Qwen3.8 Flash Next target with its paired DFlash2 drafter and a tested fresh Python installer. On the tested M1 Max, warm code and math probes decoded at 37.62 and 33.69 tokens/s, approximately 1.70× and 1.56× their matched plain baselines. Text gained much less.
This is the fixed greedy, text-only edition: thinking, images, tools, sampling and prefix caching are not exposed by its entrypoint. Plan for a 64 GiB-class Mac. The complete model download is 74.048 GB, with about 1 GB more disk space for the installed Python environment.
Choose your download
| Release | Download | Choose it for |
|---|---|---|
| Optimized plain pack | 72.843 GB | Images and optional thinking through its compatible existing helper/runtime environment |
| This DFlash2 edition | 74.048 GB, plus its Python environment | The tested installer and measured code/math decode gains, using run_dflash2.py |
Both downloads contain the same optimized target payload. This edition adds the original 1,206,140,008-byte BF16 drafter and its verified runtime path. You do not need to download the plain repository as well.
Quick facts
| Item | This edition |
|---|---|
| Platform | CPython 3.13, Apple Silicon, macOS 26 or newer |
| Installation | 73 hash-locked public PyPI wheels; fresh installation tested |
| Generation | Greedy text only; thinking off; FP32 drafter parameters; block 4 |
| Context / output | 8,192 total tokens / maximum 4,096 output tokens |
| Scoped quality result | 50/50 on the fixed public GSM8K development subset; all 50 decoded responses matched the native reference |
| Measured process peak | 42.12 GiB physical footprint on an M1 Max with 64 GiB |
RAM requirements
| Unified memory | Practical guidance |
|---|---|
| 32 GB | Below even the target's 38.01 GiB resident-payload estimate |
| 48 GB | Untested and tight; the measured process peak leaves limited headroom |
| 64 GiB | Tested on an M1 Max, with one full model loaded at a time |
The 32.0 GB PLE table is already included in the download. It is mapped/read on demand rather than fully allocated as one MLX weight array. Mapped pages, context, buffers, the drafter and other applications still need memory.
The packaged 50-case run recorded 42.12 GiB sampled process physical-footprint peak and 41.25 GiB maximum per-case MLX allocator peak. These are overlapping counters, not amounts to add. They are observations, not a whole-system RAM cap or a verified 48 GB fit. Memory evidence · Run summary
Install and run with Python
Download the complete repository and keep its folders together. From the model folder, create a fresh environment outside it:
python3.13 install_runtime.py --venv ../qwen38-runtime
../qwen38-runtime/bin/python run_dflash2.py --check-only
../qwen38-runtime/bin/python run_dflash2.py \
--prompt 'Write a Python function that merges two sorted lists.'
The bundled installer was actually tested from an empty environment: 73/73 installed distribution fingerprints matched, pip check reported zero broken requirements, and installation took 42.38 seconds on the test machine. Download/cache/network conditions can change that time. Installation evidence
The locked execution packages are official MLX/MLX-Metal 0.32.4, MLX-VLM 0.7.10, Transformers 5.17.0 and Tokenizers 0.23.3. Neither Zoo nor mlx-lm is required or present in this locked environment. Public wheel URLs, versions and hashes are recorded in runtime-wheels.json and the lockfile. No locally compiled runtime binaries are shipped.
The guard checks the exact installed artifacts and permits folder relocation when they match. Changing builds or packages requires fresh validation; there is no ignore-checks flag. --check-only imports no MLX module and loads no model.
Prompt files and shorter answers
../qwen38-runtime/bin/python run_dflash2.py \
--prompt-file question.txt --max-output-tokens 1024 \
--report ../qwen38-result.json
Reports must use a fresh path outside this uploadable model folder. The runner is one-shot: each ordinary invocation loads the model again. Short requests may spend more time loading and prefilling than decoding; a decode speedup does not remove that startup cost.
Images, thinking, tools and Unsloth Studio
This entrypoint accepts plain text and fixes thinking off, greedy decoding, context 8,192, output cap up to 4,096, drafter FP32 and block 4. Images, tools, sampling, penalties and prefix caching are unsupported here. No HTTP server or tool-execution loop is bundled.
The target has been reported by the user to work in Unsloth Studio. That feedback does not establish DFlash2 integration or these speedups in Studio. DFlash2 acceleration is verified through the included run_dflash2.py path. Choose the plain release and its compatible helper when images or thinking are needed; this installer is not a verified plain Unsloth-helper installation recipe.
Measured speed and correctness
Two repeats per probe produced six matched plain/DFlash2 pairs, with identical greedy tokens and zero errors. These are warm decode medians on an M1 Max with 64 GiB; both timed arms kept the drafter resident.
| Probe | Plain tokens/s | DFlash2 tokens/s | Observed speedup |
|---|---|---|---|
| Text | 21.48 | 22.46 | 1.048× |
| Code | 22.19 | 37.62 | 1.699× |
| Math | 21.70 | 33.69 | 1.559× |
The gain depends on the prompt and how many draft tokens the target accepts. These small shared-host probes do not establish a universal speedup. Paired benchmark
The final packaged entrypoint, running in the freshly installed public-wheel environment, answered 50/50 fixed GSM8K questions correctly, with zero generation errors, parse failures or truncated responses. All 50 decoded response texts matched the native optimized reference. Total duration was 521.04 seconds, including model loading; median recorded decode speed across those 50 cases was 31.43 tokens/s. Its in-process wall time is not an isolated comparison with the native HTTP reference. Packaged-run evidence
The selection was inspected during output-budget development. This is a fixed public development subset, not the full GSM8K benchmark, a blind generalization test, general 100% accuracy or original-BF16 retention. Gold answers never enter generation prompts. The runtime also passed strict bitwise logit/cache/rollback checks and 4/4 paired text/code/math/window-boundary cases. Strict parity evidence
Earlier runtime results and the FP32 drafter choice
An earlier upstream-source runtime also passed the fixed 50-case selection. That separate result is retained as historical evidence, rather than being substituted for the packaged public-wheel result above.
A preliminary FP16 drafter path overflowed to non-finite activations. The published entrypoint uses FP32 drafter parameters; it casts them before binding the target embedding/head. The target's native FP16 path and on-disk target weights remain unchanged. The original drafter file remains BF16 on disk. This prevents the failed FP16 path from being mistaken for the tested configuration.
Source, format and license
The target derives from the approximately 360 GB BF16 orcarouter/Qwen3.8-Flash-Next-Uncensored, revision d9dbbb7f49cad93a0e656ea69200f6b58600714a. Experts use affine 2-bit/group-128 reconstruction, other projections 8-bit, PLE mapped 4-bit storage, and sensitive/vision tensors FP16. Quantization is lossy; these task checks do not measure broad accuracy retention. Payload provenance
The drafter is Inco AI's Qwen3.8-Flash-Next-DFlash2, pinned at a22a1744e85c17364dcc54a6a420af7135d4c7f6. Credit: Inco AI, DFlash 2: Keep Drafting Parallel; Chen, Liang and Liu, DFlash, ICML 2026. Attribution
- Target: Qwen Community License 1.0.
- Drafter: Apache 2.0, separate from the target's terms.
- Included runtime helpers: their bundled license, also Apache 2.0. Dependencies retain their own licenses.
Local preparation converted/fitted source weights; it did not train on personal prompts, computer files or call logs. Public GSM8K data was used for evaluation only. This does not make a claim about undisclosed upstream training data.
Verify and keep local data private
shasum -a 256 -c SHA256SUMS
Keep prompts, result reports, credentials and caches outside any folder you intend to upload. The prepared release excludes private computer paths and local call logs. Your own application may still store its inputs and outputs. The runtime attestation records the tested artifacts without requiring an old project directory.