If your main C: drive is running low on storage, you can easily save and run the 12.1 GB model on a secondary hard drive or NVMe SSD (such as D:, E:, or G:).

Method 1: Check Available Drive Letters via Graphical Interface (GUI)

1. On your keyboard, press the Windows Key + E to open File Explorer.
2. In the left navigation pane, click on This PC.
3. Under the Devices and drives section, look at all your connected drives.
4. Note which drive letter has at least 20 GB of free space (e.g. D:, E:, or G:).

Method 2: Check Available Drive Letters via PowerShell

1. In PowerShell, type this command and press Enter:

Get-Volume | Select-Object DriveLetter, FileSystemLabel, @{Name="FreeSpaceGB";Expression={[math]::round($_.SizeRemaining/1GB,2)}}

2. You will see a clean list of every drive letter and exactly how many gigabytes of free space remain on each.

How to Apply Your Chosen Drive Letter

Whenever you see C:\QwenModel in the instructions below, simply replace the letter C with your target drive letter.

For example, if using drive D::

1. Switch to drive D in PowerShell: Type D: and press Enter.
2. Create your folder: mkdir QwenModel; cd QwenModel.
3. In docker-compose.yml, change "C:/QwenModel:/models:ro" to "D:/QwenModel:/models:ro".
4. In direct launch commands, change "C:\QwenModel\..." to "D:\QwenModel\...".

Follow these steps on Windows 10 or Windows 11:

Step 1: Open PowerShell

1. On your keyboard, press the Windows Key + R at the same time to open the Run window.
2. In the box, type: powershell and press Enter (or click OK).
3. A blue or black terminal window will appear.

Step 2: Create a Dedicated Folder

(Note: If you want to use a different drive such as D: or G:, see the Drive Selection guide above).

Copy and paste this command into your PowerShell window, then press Enter:

mkdir C:\QwenModel; cd C:\QwenModel

Step 3: Download the Model Files

Copy and paste these commands into PowerShell to download the model directly:

# Install the official downloader (if not already installed)
pip install -U huggingface_hub

# Download the model and vision files into your folder
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf --local-dir C:\QwenModel
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF mmproj-BF16.gguf --local-dir C:\QwenModel

(Alternative: You can also click the Files and versions tab at the top of this Hugging Face page and click the download button next to each file, then move them into your folder).

Step 4: Run the Server (Choose Option A or Option B)

Option A: 1-Click Launch with Docker (Recommended)

If you have Docker Desktop installed:

1. In C:\QwenModel, create a text file named docker-compose.yml.
2. Paste the following text into the file and save it (update C:/QwenModel if using another drive):

services:
  qwen38-server:
    image: ghcr.io/ggml-org/llama.cpp:server-cuda
    container_name: qwen38-server
    restart: unless-stopped
    ports:
      - "6969:6969"
    volumes:
      - "C:/QwenModel:/models:ro"
    command: >
      --model "/models/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf"
      --mmproj "/models/mmproj-BF16.gguf"
      --n-gpu-layers 99
      --ctx-size 131072
      --cache-type-k q8_0
      --cache-type-v q4_0
      --temp 1.0
      --top-p 0.95
      --top-k 20
      --min-p 0.05
      --dry-multiplier 0.6
      --dry-base 1.75
      --dry-allowed-length 2
      --xtc-probability 0.1
      --flash-attn on
      --spec-type draft-mtp
      --spec-draft-n-max 2
      -b 4096
      -ub 1024
      --cont-batching
      --parallel 1
      --reasoning-preserve
      --reasoning-budget 1024
      --metrics
      --host 0.0.0.0
      --port 6969
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

3. In your PowerShell window inside C:\QwenModel, run:

docker compose up -d

Option B: Run Directly on Windows (No Docker)

1. Download the pre-built Windows CUDA ZIP from llama.cpp Releases (look for llama-b*-bin-win-cuda-cu12.4-x64.zip).
2. Extract the files into C:\QwenModel.
3. In your PowerShell window, run:

.\llama-server.exe `
  -m "C:\QwenModel\Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf" `
  --mmproj "C:\QwenModel\mmproj-BF16.gguf" `
  -ngl 99 `
  -c 131072 `
  --cache-type-k q8_0 `
  --cache-type-v q4_0 `
  --temp 1.0 `
  --top-p 0.95 `
  --top-k 20 `
  --min-p 0.05 `
  --dry-multiplier 0.6 `
  --dry-base 1.75 `
  --dry-allowed-length 2 `
  --xtc-probability 0.1 `
  --flash-attn on `
  --spec-type draft-mtp `
  --spec-draft-n-max 2 `
  -b 4096 `
  -ub 1024 `
  --cont-batching `
  --reasoning-preserve `
  --reasoning-budget 1024 `
  --port 6969

Step 5: Start Chatting!

Open your web browser (Chrome, Edge, Firefox) and go to:
👉 http://localhost:6969

You will see the interactive chat window. Type a message and watch the model reason inside <think> tags and generate code!

Follow these steps on Apple Silicon Macs (16GB+ Unified Memory recommended):

Step 1: Open Terminal

1. On your Mac keyboard, press Command (⌘) + Space to open Spotlight.
2. Type Terminal and press Return.

Step 2: Create a Folder for Your Models

Copy and paste this into Terminal and press Return:

mkdir -p ~/QwenModel && cd ~/QwenModel

Step 3: Download the Model Files

Run this in Terminal to download the files:

curl -L -O https://huggingface.co/DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF/resolve/main/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf
curl -L -O https://huggingface.co/DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF/resolve/main/mmproj-BF16.gguf

Step 4: Install and Run llama.cpp with Apple Metal GPU

1. Install llama.cpp using Homebrew:

brew install llama.cpp

2. Start the server with full Apple Metal GPU acceleration:

llama-server \
  -m ~/QwenModel/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf \
  --mmproj ~/QwenModel/mmproj-BF16.gguf \
  -ngl 99 \
  -c 32768 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  --min-p 0.05 \
  --flash-attn on \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --reasoning-preserve \
  --reasoning-budget 1024 \
  --port 6969

Step 5: Start Chatting!

Open Safari or Chrome and navigate to:
👉 http://localhost:6969

Follow these steps on Linux with an NVIDIA GPU:

Step 1: Open Terminal

Press Ctrl + Alt + T on your keyboard to open the terminal.

Step 2: Create Folder & Download Files

mkdir -p ~/QwenModel && cd ~/QwenModel

# Download model weights
pip install -U huggingface_hub
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf --local-dir ~/QwenModel
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF mmproj-BF16.gguf --local-dir ~/QwenModel

Step 3: Run with Docker Compose

1. Create docker-compose.yml in ~/QwenModel:

services:
  qwen38-server:
    image: ghcr.io/ggml-org/llama.cpp:server-cuda
    container_name: qwen38-server
    restart: unless-stopped
    ports:
      - "6969:6969"
    volumes:
      - "$HOME/QwenModel:/models:ro"
    command: >
      --model "/models/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf"
      --mmproj "/models/mmproj-BF16.gguf"
      --n-gpu-layers 99
      --ctx-size 131072
      --cache-type-k q8_0
      --cache-type-v q4_0
      --temp 1.0
      --top-p 0.95
      --top-k 20
      --min-p 0.05
      --dry-multiplier 0.6
      --dry-base 1.75
      --dry-allowed-length 2
      --xtc-probability 0.1
      --flash-attn on
      --spec-type draft-mtp
      --spec-draft-n-max 2
      -b 4096
      -ub 1024
      --cont-batching
      --parallel 1
      --reasoning-preserve
      --reasoning-budget 1024
      --metrics
      --host 0.0.0.0
      --port 6969
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

2. Start the container:

docker compose up -d

Step 4: Start Chatting!

Open your browser and navigate to http://localhost:6969.

Because the server provides an OpenAI-compatible API endpoint at http://localhost:6969/v1, you can connect your favorite coding editors:

In ZCode

1. Open ZCode Settings -> Add Custom Model.
2. Provider: OpenAI-compatible.
3. Base URL: http://127.0.0.1:6969/v1
4. API Key: local (or leave blank).
5. Model Name: Qwen3.8-27B-Uncensored
6. Context Window: 131072

In Cursor

1. Open Cursor Settings -> Models.
2. Under OpenAI API Key, type: local.
3. Under Base URL, type: http://localhost:6969/v1.
4. Add Model Name: Qwen3.8-27B-Uncensored.

In Zed

1. In Zed Settings (settings.json), add a custom endpoint:

{
  "language_models": {
    "openai": {
      "api_url": "http://localhost:6969/v1",
      "available_models": [
        {
          "name": "Qwen3.8-27B-Uncensored",
          "max_tokens": 131072
        }
      ]
    }
  }
}