license: mit
library_name: mlx
base_model: llmfan46/Ornith-1.0-35B-uncensored-heretic
tags:
- mlx
- mlx-vlm
- moe
- multimodal
- vision
- coding
- agentic
- uncensored
- heretic
pipeline_tag: image-text-to-text
language: - zh
- en
English | 中文
English
This model was converted to MLX format from llmfan46/Ornith-1.0-35B-uncensored-heretic.
Subjective Evaluation
- This model has some playability and is worth trying.
Things To Note
- I ran some tests, HumanEval's score may be a little low (More obvious), This model may see more retries in some agent tasks that involve writing short scripts.
- I've noticed that some of my cron jobs are taking longer to run than they used to.（Hermes-Agent）
- Ornith-1.0-35B-uncensored-heretic-MLX-4bit:
~66.7%(this model) - Ornith-1.0-35B-4bit:
~86.7% - Software:
oMLX-v0.4.4 - Sample Size:
30
Recommended Inference Parameters
01
| Parameter | Value |
|---|---|
temperature |
0.6 |
top_p |
0.95 |
top_k |
20 |
repeat_penalty |
1.05 |
02
| Parameter | Value |
|---|---|
temperature |
1.0 |
top_p |
1.0 |
top_k |
40 |
min_p |
0.01 |
repeat_penalty |
1.05 |
- When the value of
repeat_penaltyis set to1.0, the model is more prone to thinking loops. - When the value of
repeat_penaltyis set to1.08or greater, in certain tasks, it is easier to change the English period of some file suffix names into Chinese periods, thereby affecting tool invocation. - Other parameters have not been fully verified and are currently for reference only.
Code To Quantify This Model
from mlx_vlm import convert # mlx_vlm >= 0.6.4
convert(
hf_path="/Volumes/original_model_path/Ornith-1.0-35B-uncensored-heretic",
mlx_path="/Volumes/quantized_model_path/Ornith-1.0-35B-uncensored-heretic-MLX-4bit",
quantize=True,
q_bits=4,
q_group_size=32
)
中文（Chinese）
该模型从 llmfan46/Ornith-1.0-35B-uncensored-heretic 转换为 MLX 格式。
主观评价
- 这个模型有一定的可玩性，值得尝试。
注意事项
- 我进行了一些测试，HumanEval 的分数可能偏低（比较明显）。在一些涉及编写短脚本的 Agent 任务中，该模型可能需要更多重试。
- 我注意到我的一些 cron 作业运行时间比过去更长了（Hermes-Agent）。
- Ornith-1.0-35B-uncensored-heretic-MLX-4bit：
~66.7%（此模型） - Ornith-1.0-35B-4bit：
~86.7% - 软件：
oMLX-v0.4.4 - 样本量：
30
推荐的参数
01
| 参数 | 值 |
|---|---|
temperature |
0.6 |
top_p |
0.95 |
top_k |
20 |
repeat_penalty |
1.05 |
02
| 参数 | 值 |
|---|---|
temperature |
1.0 |
top_p |
1.0 |
top_k |
40 |
min_p |
0.01 |
repeat_penalty |
1.05 |
- 当
repeat_penalty的值设置为1.0时，模型更容易出现思维循环。 - 当
repeat_penalty的值设置为1.08或更大时，在某些任务中，更容易将某些文件后缀名的英文句点更改为中文句点，从而影响工具调用。 - 其他参数尚未完全验证，目前仅供参考。
其他规格
6bit：经测试，平均性能提升 ~3%（由于我的流量不多了，所以暂时还没上传）
量化此模型的代码
from mlx_vlm import convert # mlx_vlm >= 0.6.4
convert(
hf_path="/Volumes/original_model_path/Ornith-1.0-35B-uncensored-heretic",
mlx_path="/Volumes/quantized_model_path/Ornith-1.0-35B-uncensored-heretic-MLX-4bit",
quantize=True,
q_bits=4,
q_group_size=32
)