language:

• zh
• en
  license: other
  tags:
• gguf
• MoE
• financial-llm
• MoziSmartBit
• qwen3.5
• qwen3.6
• ornith
• MoziAI
• tool-calling
• uncensored
• vision
• mtp
• a3b
  library_name: llama-cpp
  pipeline_tag: text-generation

MoziAI-35B-A3B-MOE-MTP-Uncensored - Free Locally Deployable Small Yet Powerful Multimodal AI

Language / 语言选择
English | 简体中文 | 繁體中文 | 日本語 | 한국어 | हिन्दी | Deutsch | Français | Nederlands | Italiano | Русский

Model Overview

MoziAI-35B-A3B-MOE is a local open-source multimodal AI large language model developed by Chinese finance influencer Chen Yumo's team (focuses on financial domain, supports vision, tool calling, complex long tasks, consumer GPU local deployment). It is fine-tuned/distilled from the Ornith-1.0-35B-A3B (Qwen3.5-35B-A3B/Qwen3.6-35B-A3B architecture, MIT licensed) foundation model.

Our team's mission is to bring powerful local AI large models to thousands of households and small-medium enterprises, eliminating the need for expensive AI hardware costs or cloud API costs. Through the self-developed MoziSmartBit Intelligent Quantization technology, the 35B-parameter MoE model is compressed to approximately 15.5 GB, achieving an optimal balance between precision and size with near-lossless ~99% precision quality. Although the model has 35 billion parameters in total, it uses MOE sparse expert technology that only activates 3 billion parameters per token and supports MTP speculative decoding for accelerated inference. Practical testing shows it can be deployed locally on a consumer GPU with 20GB VRAM and achieves 140+ tokens/s inference speed, outperforming many cloud-based paid AI large models.

In addition to retaining general AI capabilities, this model focuses on optimizing financial vertical domain applications, including financial Q&A, quantitative programming, general programming, tool calling, and improving success rate for 256K complex long context tasks. It can be deployed locally for free on consumer-grade GPUs, saving substantial cloud token costs, achieving 7X24 token freedom while ensuring local data privacy and security.

Release Date: 2026-08-20 | Version: V3.6

Model Features

• MoziSmartBit Intelligent Quantization: Self-developed smart quantization, best balance of precision and size, nearly lossless compressed to approximately 15.5 GB
• Complex Long Task Capability: Trained with intelligent loop mechanism that allows the agent to automatically plan tasks, handle stuck points and self-reflect, enabling automatic execution and self-adjustment for complex tasks, eliminating the need for users to continuously optimize prompts manually.
• Small Model, Big Capability: Outperforms other models within 350 billion parameters on complex tasks, even outperforms some larger models with multiple times the parameters.
• MOE+MTP Speed Advantage: Although the model has 35 billion parameters in total, it only activates 3 billion parameters (8+1 experts) per token, resulting in faster inference speed. Perfect for local deployment on consumer GPUs with 20GB or 24GB VRAM, delivering 140+ tokens/s inference speed.
• Financial Vertical Focus: Deep optimization for financial Q&A, quantitative programming, and tool calling
• Consumer-grade Deployment: Deployable on consumer GPUs with 20GB or 24GB+ VRAM, supports 256K long context
• Multilingual Support: 201 languages and dialects, with enhanced Chinese capabilities, covering English/Japanese/Korean/German/French/Spanish/Portuguese and more
• General Programming: Full-stack development, code debugging, architecture design, script writing, covering Python/JS/TS/Go/Rust and other mainstream languages
• Article Writing: High-quality multi-genre writing including research reports, analysis articles, technical documentation, creative content
• Vision Understanding: Supports multimodal vision by loading vision file in inference framework, local screenshot can be directly pasted into chat window for image comprehension
• Uncensored Free Output: No content censorship, free discussion on any topic without safety restrictions
• Enhanced Reasoning: Chain-of-thought training for improved reasoning quality
• Multi-Framework Support: Compatible with llama.cpp, Ollama, LM Studio, Jan
• Multi-Agent Platform Support: Deep integration with OpenClaw, Hermes, OpenCode, Cursor, Windsurf, Claude Code, Codex and other mainstream AI IDEs and Agent frameworks, natively supports tool calling and multi-turn task orchestration, ready to use out of the box

Uncensored Advantages

This model inherits the Uncensored feature from the Ornith-1.0-35B base model, with the following advantages:

Use Cases: Academic research, deep analysis, free discussion, unrestricted AI conversation.
Note: This is a locally deployed model, output content is fully controlled by the user, no content moderation responsibility.

Core Capabilities

Technical Specifications

Quantization Format & Model Size Comparison

MoziAI V3.6 uses MoziSmartBit Intelligent Quantization, maintaining ~99% precision while compressing the 35B parameter MoE model to ~15.5 GB (~4.5x compression ratio), balancing inference quality with deployment accessibility for consumer GPUs.

MoziSmartBit Intelligent Quantization

Traditional quantization uses uniform precision across all layers. MoziSmartBit Intelligent Quantization applies differentiated quantization strategies for optimal size-precision balance.

Compression Effect

Traditional quantization compresses all parts of the model uniformly, often leading to significant precision loss. MoziSmartBit Intelligent Quantization uses a self-developed intelligent compression strategy that achieves substantial size reduction with minimal precision loss:

• Minimal Quantization Loss: Training gains > quantization loss. The trained MoziAI-35B achieves better PPL on financial domain text than the pre-training bf16 base model, reducing hallucination and perplexity compared to similar AI models
• ~4.5x Size Reduction: Compressed from ~70 GB (FP16) to ~15.5 GB, also significantly smaller than Q4_K_M (~21 GB), significantly lowering VRAM and storage requirements
• Consumer GPU Friendly: A 35B MoE model that previously required high-end GPUs can now run smoothly on 20GB~24GB VRAM

Comparative Advantages

vs Q4_K_M (~22 GB): ~30% smaller (~15.5 GB), with precision higher than Q4_K_M, lower VRAM barrier. Runs smoothly on mid-range consumer GPUs (24GB).

vs FP16 original (~70 GB): ~4.5x compression, training effective + minimal quantization loss (training gains > quantization loss), enabling local 256K context deployment on consumer GPUs instead of professional-grade hardware.

Recommended Inference Parameters

Based on local production config (AMD Radeon AI PRO R9700 32GB):

llama.cpp Launch Command

llama-server \
  -m V3.6/moziAI-V3.6-Qwen3.6-35B-A3B-Ornith-MoziSmartBit-Q4_K_M-Uncensored.gguf \
  --mmproj V3.6/moziAI-V3.6-35B-uncensored-heretic-mmproj-BF16.gguf \
  --chat-template-file V3.6/moziAI-V3.6-35B-chat-template.jinja \
  -c 262144 -ngl 99 -t 28 \
  --batch-size 2048 --ubatch-size 512 \
  --flash-attn auto \
  --cache-type-k q4_0 --cache-type-v q4_0 --kv-unified \
  --poll 0 --reasoning on --reasoning-budget 400 \
  --host 0.0.0.0 --port 8080 \
  --temp 0.6 --top-p 0.95 --top-k 20

VRAM Configuration Recommendations

Since user GPU configurations vary widely, here are recommended parameters for different VRAM sizes (all for MoziSmartBit version):

NVIDIA

AMD

Intel

Shared Memory iGPUs

💡 Tip: As long as your VRAM meets the above requirements, it works. No brand or model restrictions. Supports NVIDIA / AMD / Intel discrete GPUs, and also 128GB unified memory iGPUs listed above.

💡 Tip: Longer context uses more VRAM. If you encounter OOM (out of memory), gradually reduce the -c value. Use --fit on to let llama.cpp auto-adjust layers to fit your VRAM.

Ollama Deployment

# Create Modelfile
FROM ./moziAI-V3.6-Qwen3.6-35B-A3B-Ornith-MoziSmartBit-Q4_K_M-Uncensored.gguf

PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER num_ctx 262144
PARAMETER num_gpu 99

# Build and run
ollama create moziAI-35B -f Modelfile
ollama run moziAI-35B

LM Studio / Jan Deployment

Search moziAI-35B in LM Studio or Jan, download the MoziSmartBit quant version.

Benchmarks

MoziAI-35B-V3.6 is fine-tuned from Ornith-1.0-35B (deepreinforce-ai). MoziAI adds financial vertical domain optimization on top of the excellent agentic coding capabilities of the base model, delivering superior performance in financial Q&A, quantitative programming, and tool calling scenarios. General capabilities remain consistent with the Ornith-1.0-35B base model.

• Terminal-Bench 2.1 (Terminus-2): Evaluated using the Harbor/Terminus-2 framework with parser=json, temperature=1.0, top_p=1.0, and a 128K context window. Each run uses a 4-hour timeout with 32 CPU cores and 48GB RAM, results averaged over 5 runs.
• Terminal-Bench 2.1 (Claude Code): Evaluated using Claude Code 2.1.126 with parser=json, temperature=1.0, top_p=1.0, max_new_tokens=131072. Results averaged over 5 runs.
• SWE-bench Verified, Pro and Multilingual: Evaluated using OpenHands harness with temp=1.0, top_p=0.95, 256K context window.
• NL2Repo: Evaluated with temperature=1.0, top_p=1.0, 400K context, 48K output.

MoziAI-35B inherits all the excellent agentic coding capabilities from Ornith-1.0-35B. MoziAI's core differentiation is deep optimization for financial vertical domains, significantly outperforming general models in scenarios like financial report analysis, quantitative strategy, risk & compliance, and agent tool calling.

Model Download

Due to the large model size (~15.5 GB), weights are hosted on multiple community platforms:

💡 LM Studio: You can also search and download directly in LM Studio. Search moziAI in the Discover tab and click Download.
💡 Download Tip: Click the link above to go to the HuggingFace repository, then go to the "Files and versions" tab to download all files under the V3.6 directory (main model, vision projection, chat template). Make sure all three files are placed in the same directory.

⚠️ Important: Vision Capability Requires mmproj File

This model supports multimodal vision. The vision projection file (mmproj) is included in the version directory:

• Vision file: moziAI-V3.6-35B-uncensored-heretic-mmproj-BF16.gguf (~903 MB, BF16 precision)
• Placement: Same version directory as the GGUF model file
• Loading: Load with --mmproj flag when starting llama-server

llama-server -m V3.6/moziAI-V3.6-Qwen3.6-35B-A3B-Ornith-MoziSmartBit-Q4_K_M-Uncensored.gguf \
  --mmproj V3.6/moziAI-V3.6-35B-uncensored-heretic-mmproj-BF16.gguf

Without the vision file, the model will lose image understanding capability and only retain text-only conversation.

Quick Start

1. Download Model Files

Download all files under the V3.6 directory from HuggingFace / ModelScope:

V3.6/
├── moziAI-V3.6-Qwen3.6-35B-A3B-Ornith-MoziSmartBit-Q4_K_M-Uncensored.gguf      # Main model (required)
├── moziAI-V3.6-35B-uncensored-heretic-mmproj-BF16.gguf  # Vision projection (optional)
└── moziAI-V3.6-35B-chat-template.jinja                  # Chat template (recommended)

2. Start Inference Server

For the full recommended configuration, see llama.cpp Launch Command above.

Minimal launch (core params only):

llama-server \
  -m V3.6/moziAI-V3.6-Qwen3.6-35B-A3B-Ornith-MoziSmartBit-Q4_K_M-Uncensored.gguf \
  --chat-template-file V3.6/moziAI-V3.6-35B-chat-template.jinja \
  -c 262144 -ngl 99

Add --mmproj V3.6/moziAI-V3.6-35B-uncensored-heretic-mmproj-BF16.gguf for vision capability.

Transformer-based Inference

MoziAI-35B-V3.6 is based on the Ornith-1.0-35B architecture and is compatible with modern transformer inference frameworks. Serving requires recent runtimes:

• Transformers ≥ 5.8.1
• vLLM ≥ 0.19.1
• SGLang ≥ 0.5.9

vLLM

vllm serve chenyumo/moziAI-35B-A3B-MOE-MTP-Uncensored \
    --served-model-name MoziAI-35B \
    --tensor-parallel-size 8 \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.90 \
    --enable-prefix-caching \
    --enable-auto-tool-choice --tool-call-parser qwen3_xml \
    --reasoning-parser qwen3 \
    --trust-remote-code

SGLang

python -m sglang.launch_server \
    --model-path chenyumo/moziAI-35B-A3B-MOE-MTP-Uncensored \
    --served-model-name MoziAI-35B \
    --tp 8 \
    --host 0.0.0.0 --port 8000 \
    --context-length 262144 \
    --mem-fraction-static 0.85 \
    --tool-call-parser qwen3_coder \
    --reasoning-parser qwen3

Hugging Face Transformers

For a quick local test, load the model directly with Transformers. Make sure you have a recent release:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "chenyumo/moziAI-35B-A3B-MOE-MTP-Uncensored"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a Python function is_prime(n). Keep it short."}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
generated = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
)
output_ids = generated[0][inputs.input_ids.shape[1]:]

# The reply contains a <think> ... </think> reasoning block followed by the answer.
content = tokenizer.decode(output_ids, skip_special_tokens=True)
if "</think>" in content:
    reasoning, answer = content.split("</think>", 1)
    reasoning = reasoning.replace("<think>", "").strip()
    answer = answer.strip()
else:
    reasoning, answer = "", content.strip()

print("Reasoning:", reasoning)
print("\nAnswer:", answer)

3. Start Using

Open http://localhost:8080 in your browser to start chatting.

Directory Structure

moziAI-35B/
├── README.md              # This file (English)
├── README.zh.md           # Chinese version
├── LICENSE                # License
├── V3.6/                  # V3.6 version (self-contained)
│   ├── RELEASE_NOTES.md                       # Release notes
│   ├── moziAI-V3.6-Qwen3.6-35B-A3B-Ornith-MoziSmartBit-Q4_K_M-Uncensored.gguf    # Main model
│   ├── moziAI-V3.6-35B-uncensored-heretic-mmproj-BF16.gguf # Vision projection
│   └── moziAI-V3.6-35B-chat-template.jinja   # Chat template

For the future upgrade plan, see Future Upgrade Plans.

SEO Keywords

financial AI LLM, local open source model, end-side model, quant programming, MoziSmartBit, intelligent quantization, GGUF quantization, MoE model, local open source LLM, local deployment, financial AI, tool calling, Agent, llama.cpp, Ollama, GGUF, Uncensored, no censorship, free output, unrestricted, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0, Ornith-1.0-35B, Qwen3.5, Qwen3.6, financial vertical domain, open source model

License (Important)

This model uses a Custom Restrictive License:

✅ Allowed

• Free Commercial Use: Free to integrate into commercial products
• Copy & Distribute: Can copy, download, and share

See LICENSE for full terms.

Disclaimer

Provided "as is" without warranty. Model output is for reference only, not investment advice. Users bear all risks.

Contact

• HuggingFace: @chenyumo
• GitHub: @chenyumo166
• Weibo: @rimochen
• E-mail: [email protected]

Copyright (c) 2026 Chen Yumo / chenyumo166. All rights reserved.