license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
base_model:
- orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
library_name: mlx-vlm
pipeline_tag: image-text-to-text
tags: - mlx
- qwen4-exp
- quantized
- vision
- apple-silicon
- experimental
Qwen3.8-Flash-Next-Uncensored-mlx-optimized-2-8bit
An Apple Silicon MLX conversion of the approximately 360 GB BF16 source, packaged in 72.843 GB with native qwen4_exp text and vision support. Use the included helper for text, images and optional thinking. It applies the FP16 PLE adapter required by this pack.
Plan for a 64 GiB-class Mac and a compatible existing Unsloth MLX environment. The model was tested on an M1 Max with 64 GiB unified memory. This release's source-build instructions are an unverified clean-install outline; the separate DFlash2 edition has its own tested installer.
Choose your download
| Release | Download | Choose it for |
|---|---|---|
| This optimized pack | 72.843 GB | Text, full-detail images and optional thinking through the compatible helper/runtime |
| Optimized DFlash2 | 74.048 GB, plus about 1 GB for its Python environment | The tested fresh installer and faster code/math decoding with fixed greedy, text-only settings |
The DFlash2 edition includes the same optimized target payload plus a drafter. Its speed gains were measured against a matched plain baseline in that edition's own tests. This plain helper uses ordinary decoding.
Quick facts
| Item | This release |
|---|---|
| Runtime | Native MLX-VLM qwen4_exp, loaded through the included Unsloth helper |
| Storage | 72.843 GB, including the 32.0 GB disk-backed PLE table |
| Resident payload estimate | 38.01 GiB; this excludes total runtime/OS memory |
| Tested helper defaults | Context 16,384; output cap 4,096; thinking off; greedy decoding |
| Images | Full detail by default: --image-max-tokens 0 |
| Scoped quality result | 50/50 on a fixed public GSM8K development subset at the 4,096-token output cap |
| Observed API decode | About 23.5 tokens/s on six small text/code probes with mixed cache states |
RAM requirements
| Unified memory | Practical guidance |
|---|---|
| 32 GB | Below the 38.01 GiB resident-payload estimate; unsuitable for the tested profile |
| 48 GB | Untested and tight; a successful fit is not established |
| 64 GiB | Tested on an M1 Max, with one full model loaded at a time |
The PLE table is already included in the download. Rows are mapped and read on demand, so its full 32.0 GB is not one resident MLX weight allocation. Mapped pages, context, image activations, buffers and other applications still consume memory.
A native run at a 16,384-token context budget recorded a 39.72 GiB process physical-footprint peak. A separate historical 8,192-context run observed 41.20 GiB under different conditions. These observations do not establish a 48 GB minimum or a simple relationship between context and peak memory. RSS, allocator memory and physical footprint overlap; do not add them. Native resource evidence · Earlier observations
Unsloth Studio
The target has been reported by the user to work in Unsloth Studio. This is user feedback, not an independently measured Studio benchmark or a promise about every Studio version.
The instructions below use the bundled Python helper. The DFlash2 speed measurements belong to its separate run_dflash2.py entrypoint; loading this target in an application does not establish speculative decoding.
Run with Python
Keep the complete repository together, including its PLE files, tokenizer, processor and helpers. Use Python 3.13 in a compatible local MLX/Unsloth environment. Read INSTALL.md before changing dependencies: it documents the tested source builds and their three declared Zoo dependency-bound conflicts. A fresh plain-helper installation has not been verified.
From the downloaded model folder:
python run_qwen38_unsloth.py --model . --check-only
python run_qwen38_unsloth.py --model . \
--prompt 'Compute 19 + 23. Reply with only the integer.'
This is a one-request helper; a new invocation loads the model again. Its defaults are context 16,384, output cap 4,096, greedy decoding and thinking disabled. Model loading and prompt processing add latency beyond decode tokens/second. Generic loaders that omit the included FP16 PLE adapter are unverified.
--patch-mode native is the default: it uses the unpatched loader and native MLX-VLM generation. --patch-mode patched opts into Unsloth inference scopes; image requests retain the native fused vision loader.
Images and small text
python run_qwen38_unsloth.py --model . \
--image screenshot.png --image-max-tokens 0 \
--prompt 'Read the small text in this screenshot.'
Cap 0 preserves the processor's full-detail image path. Large images take longer and use temporary memory; this setting adds no model-weight disk storage. A positive --image-max-tokens value reduces large-image detail. The helper uses the native fused vision loader for image requests to avoid the earlier dense-attention allocation failure.
A 2048×2048 color fixture passed at the combined 16,384-context/4,096-output profile. A separate synthetic OCR fixture returned its six-digit control code exactly. These are targeted checks, not general photo, screenshot or OCR accuracy benchmarks. Full-image check · OCR check
Thinking and tools
Enable bounded thinking explicitly when useful; its tokens share the output allowance:
python run_qwen38_unsloth.py --model . --thinking \
--thinking-budget 512 --max-tokens 1024 \
--prompt 'Explain your calculation briefly, then give the answer.'
The model/template retains tool-call capability. A native tool call and mock-result round trip passed in a separate API harness. This folder does not bundle an HTTP server or an application tool-execution loop. The included CLI is not a tool agent. API checks
What was tested
The 50/50 result used the same fixed public GSM8K selection, greedy decoding, thinking off and a 4,096-token output cap, with zero API errors, parse failures or truncated generations. The questions were inspected during budget/truncation development. This is a development subset, not a blind full-benchmark result, general 100% accuracy or an original-BF16 retention measurement. Report
Separate API probes measured medians of 23.47 tokens/s for explanation and 23.49 tokens/s for Python, three requests each. Cache states were mixed. These are decode observations, not cold-start latency or a universal speed claim. Timing evidence
Earlier results, failures and test scope
- The earlier fixed-subset scores were 45/50 (90%) at 384 output tokens, 49/50 (98%) at 1,024, and 50/50 at 4,096. Parse failures and truncations remain in the denominator; the budgets are separate measurements. 384 · 1024 · 4096
- The native API smoke suite passed 15/18 arithmetic/fact checks. Its structured-output, tool-round-trip and vision checks passed. Smoke evidence
- Longer-prompt/image integration completed three responses with zero errors, but passed 2/3 strict checks: the lookup answer was correct but its JSON had Markdown fences. The image and follow-up checks passed. Integration evidence
- The first patched-loader 2048×2048 image attempt failed with a Metal out-of-memory error. The helper subsequently used the native fused vision loader and passed the combined full-image profile. The failed attempt is retained. Failure · Completed check
- Six malformed client-generated PNGs were rejected before inference. Corrected PIL-verified color fixtures completed without API errors. Flat-color requests do not measure OCR accuracy. Separate API resolution/follow-up timings also depend on cache reuse, which the one-shot CLI does not provide. Resolution evidence · Warm/follow-up evidence
- One matched native/patched Unsloth text prompt produced identical output. Its reported rates were 20.38 and 26.68 tokens/s respectively; a single prompt does not establish general parity or a general speed improvement. Comparison
Format, source and limitations
Experts use affine 2-bit/group-128 reconstruction fitting; other projections use 8-bit/group-64. PLE uses mapped 4-bit/group-32 storage, while sensitive components and vision weights are FP16. MTP weights are omitted. Quantization is lossy; reconstruction error and these small task results do not establish broad accuracy retention. Conversion record
The source is orcarouter/Qwen3.8-Flash-Next-Uncensored, revision d9dbbb7f49cad93a0e656ea69200f6b58600714a. Local preparation converted and fitted source weights; it did not train on personal prompts, computer files or call logs. Public GSM8K questions were used for evaluation only. Undisclosed upstream training data is outside that claim. Source attribution
License and file verification
The target is governed by Qwen Community License 1.0. Read its terms before use or redistribution. Runtime dependencies retain their respective licenses. The public GSM8K evaluation license is included separately.
shasum -a 256 -c SHA256SUMS
Keep local prompts, result reports, caches and credentials outside any folder you intend to upload. The prepared publication contains portable inference files, public evaluation evidence and provenance; this does not control the logs created by your own application.