base_model:
- orcarouter/Qwen3.8-27B-Uncensored
- harness-hub/Qwen3.8-27B-hermes-adapted
library_name: llama.cpp
license: apache-2.0
language: - en
- zh
pipeline_tag: image-text-to-text
tags: - gguf
- qwen3.8
- qwen3_5
- abliterated
- uncensored
- hermes
- lora-merge
- agentic
- tool-calling
- llama.cpp
Qwen3.8-27B-Uncensored + Hermes-adapted (Q4_K_M)
The uncensored (abliterated) Qwen3.8-27B, with harness-hub's Hermes RL adapter fused into the weights, quantized to
Q4_K_M for local use on a single machine.
The point of this merge: keep the refusal-free behaviour of the abliterated base, while gaining the adapter's
agentic discipline — fewer wrong turns, fewer tool calls, far fewer generated tokens per task under Hermes Agent.
Files
| File | Size | Notes |
|---|---|---|
Qwen3.8-27B-Uncensored-hermes-Q4_K_M.gguf |
16.0 GiB | Main model. Text + tool calling. |
mmproj-Qwen3.8-27B-Uncensored-hermes-F16.gguf |
0.93 GiB | Vision projector. Unmodified — abliteration and the LoRA touch language-model weights only. |
The LoRA does not train or alter the vision tower or the MTP head, so this model is vision-capable exactly as the
base is, via the projector above.
How it was made
The adapter is a PEFT LoRA (r=32, α=64, scale 2.0) over Qwen/Qwen3.8-27B, targeting q/k/v/o_proj andgate/up/down_proj. Those targets exist in 16 attention layers and all 64 main FFN layers of Qwen3.8-27B's hybrid
SSM/attention stack; the 48 linear-attention (SSM) blocks expose no matching module names, and the MTP head at
layer 64 is outside the adapter — so neither was touched. That coverage is inherited from the adapter, not a choice
made here.
# 1. adapter -> GGUF
python convert_lora_to_gguf.py \
--base-model-id Qwen/Qwen3.8-27B \
--outfile hermes-lora-f16.gguf --outtype f16 ./lora
# 2. fuse onto an 8-bit copy of the uncensored base (higher-precision starting point than Q4)
llama-export-lora \
-m orcarouter_Qwen3.8-27B-Uncensored-Q8_0.gguf \
--lora hermes-lora-f16.gguf \
-o merged.gguf
# 3. quantize
llama-quantize --allow-requantize merged.gguf \
Qwen3.8-27B-Uncensored-hermes-Q4_K_M.gguf Q4_K_M
Q8_0 was deliberately used as the fusion base instead of the 4-bit file: the LoRA delta is small compared to the
weights (≈0.15–0.25% RMS), so fusing onto already-4-bit weights would have buried a good part of it under
quantization noise.
Usage
llama.cpp:
llama-server \
-m Qwen3.8-27B-Uncensored-hermes-Q4_K_M.gguf \
--mmproj mmproj-Qwen3.8-27B-Uncensored-hermes-F16.gguf \
--ctx-size 131072 --flash-attn on \
--cache-type-k q8_0 --cache-type-v q8_0 \
--reasoning off
--reasoning off matters. The upstream card is explicit that this adapter is trained for thinking-off use: thinking-on
is within noise of thinking-off for the adapter and costs roughly 3× the tokens. On this base, leaving thinking on is
the single largest latency cost per reply.
LM Studio: load the Q4_K_M file, attach the mmproj file for image input, disable thinking in the model settings.
What to expect, and what not to
- Uncensored behaviour comes from the abliterated base weights, which are present here in full. Nothing in this
merge restores refusals. - The adapter's published numbers (SWE-bench Pro 43.1%, Terminal-Bench 2.1 58.8% thinking-off) were measured on the
stock, non-abliterated base at BF16 / 131k context. Abliteration edits the same residual-stream components the
adapter was trained against, so expect a partial, not exact, transfer of those gains. This specific combination has
not been benchmarked. - Q4_K_M dilutes the adapter somewhat. A rank-32 delta at ~0.2% of weight RMS sits close to 4-bit quantization
noise. Behaviour tracks the base + adapter, but a Q6_K or Q8_0 build of the same merge would be measurably closer to
the intended model if you have the memory. - Not evaluated by the uploader beyond smoke tests (loads, answers, no reasoning tokens leaked at
--reasoning off,
coherent coding answers, merge deltas verified numerically againstB·A·α/r).
Context
The base supports long context; 131k is a sane default for 48 GB unified memory with q8_0 KV cache. 262k works but
prefill cost and memory pressure rise sharply, and long contexts dilute attention.
Credits
- Base weights: orcarouter/Qwen3.8-27B-Uncensored (abliterated Qwen3.8-27B, Apache-2.0)
- Adapter: harness-hub/Qwen3.8-27B-hermes-adapted (Apache-2.0)
- 8-bit fusion source: bartowski/orcarouter_Qwen3.8-27B-Uncensored-GGUF
- Upstream model: Qwen/Qwen3.8-27B
Apache-2.0, inherited. Abliterated models carry no safety tuning — you are responsible for what you run them on.