library_name: mlx
pipeline_tag: text-generation
base_model: benjamin920101/Qwen3.8-9B-Distill-uncensored-heretic-china_uncensored_dpo
tags:

• mlx
• 4-bit
• qwen3_5

Qwen3.8 9B Distill — MLX 4-bit

Converted from benjamin920101/Qwen3.8-9B-Distill-uncensored-heretic-china_uncensored_dpo at revision 9fbf58a1dc50b48c87b465f45c403db5a40ed8e8.
Upstream naming is preserved; the underlying architecture is Qwen3.5 text (qwen3_5_text).

Conversion

• Original BF16 checkpoint converted directly to MLX affine 4-bit, group size 64.
• Upstream qwen3_5_text model type mapped to the MLX implementation qwen3_5.
• No additional training or changes to learned weights apart from quantization.
• The upstream config declares one MTP layer, but its weight index contains no MTP tensors.
  This conversion sets mtp_num_hidden_layers to zero. It does not provide MTP or fabricate an MTP layer.
• Source attribution is preserved. No new license is asserted; review upstream rights before redistribution.
• A short deterministic inference smoke test was run; see conversion-report.json.
  This is not a quality, safety, or benchmark evaluation.

Usage

from mlx_lm import load, generate
model, tokenizer = load("PATH_OR_REPOSITORY")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello"}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=128))

For LM Studio, use its MLX runtime. Start with a moderate context length (e.g. 8192)
on 24 GiB machines rather than the upstream 262144-token maximum.