license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- v14-episodic-memory
- sglang
- fp8
- heretic
- qwen3_8
- qwen3_6
- uncensored
- finetune
- Cold Fusion
- GAIN Training
- Multi-stage tuning
- all use cases
- unsloth
datasets: - DavidAU/Polar-STRICT-Datasets
- DavidAU/F451-STRICT-Datasets
- DavidAU/THE-DECKARD-Datasets
language: - ar
- zh
- en
- fr
- de
- hi
- id
- it
- ja
- ko
- pl
- pt
- ru
- es
- th
- vi
base_model: - DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored
base_model_relation: quantized
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14-FP8
繁體中文摘要:這是 DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored 的 V14 轉換版(LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14)量化成 FP8 的版本,格式與 Qwen 官方的 FP8 版相同。權重 28.5 GB(bf16 版 51 GB),用 Docker 鏡像 lovewin10/sglang-v14:nightly-dev-cu13-20260924-ffac53d7-v14(修改過的 SGLang)服務時 decode 從每秒 48.5 提高到 71.5 個 token,記憶的各項檢查與 bf16 版相同,困惑度高 0.30%。模型帶有跨請求保存的情節記憶:在一段對話裡說過的事,之後在另一段全新的對話裡問得到;CUDA graph、前綴快取與推測解碼照常開著。實測結果與已知的限制寫在下面。
DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored converted to the V14 episodic memory architecture (Qwen3_5V14ForConditionalGeneration) and quantized to FP8.
The weights are the bf16 conversion's (LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14, the original's byte for byte) quantized to FP8 in the format of Qwen's own FP8 releases (Qwen/Qwen3.5-27B-FP8): E4M3 weights in 128×128 blocks with one bf16 factor per block (
weight_scale_inv), activations quantized by the engine at run time. Nothing was trained or merged.Added, as in the bf16 conversion: the architecture name, an
auto_mapand the memory options (qwen_dense_v14_memory_config) inconfig.json, and two code files (configuration_qwen_dense_v14_reinstatement.py,modeling_qwen_dense_v14_reinstatement.py) that hold the memory; and thequantization_config.Served by the Docker image
lovewin10/sglang-v14:nightly-dev-cu13-20260924-ffac53d7-v14, a modified SGLang, the model has a memory that persists across requests: what was said in one chat is answered in a later, separate chat. The store is on disk and survives restarts.407 weights are FP8: the attention q/k/v/o projections, the MLP, the Gated DeltaNet
in_proj_qkv,in_proj_zandout_proj, and those of the MTP layer. Kept in bf16, as Qwen'smodules_to_not_convertlists them: the embeddings,lm_head, every norm, the Gated DeltaNetconv1d,in_proj_aandin_proj_b,mtp.fcand the vision tower. 28.75 GiB of weights; a quantized weight differs from the bf16 one by 2.65% on average and 2.648% at most (relative Frobenius error).The memory names its store after the weights, so this directory opens a store of its own: it does not read a store the bf16 conversion wrote, nor the other way round.
model-mtp-restored.safetensors(from the source repository) holds the MTP head the speculative decoding of the command below drafts with.chat_template_fixed.jinjais an additional chat template from the local copy the conversion was made from; the command below useschat_template.jinja, the source repository's.
How the memory works
While a prompt is read, the keys and values its attention layers compute are cut into events of up to 128 tokens and written to a store on disk, each with a low-rank engram of the state it leaves in the model's recurrent layers. A later prompt is read once to see what it asks for; the stored events whose keys would take its attention are recalled by one vote for all layers and heads; their engrams are composed into the request's recurrent state and the prompt is read again with the recalled keys and values merged into the attention. Nothing is retrieved as text and nothing is added to the prompt. A question that asks for nothing in the store recalls little or nothing, but its answer is not always the plain model's word for word: a stored event that resembles it can pass the vote, and while generating the request also attends the event the memory's world model expects next (foresight).
Serve with SGLang (Docker)
Needs one 96 GB GPU (the arguments below were run on an RTX PRO 6000 Blackwell; the weights take 28.5 GB of it), Docker with the NVIDIA container runtime, and a writable directory for the memory store.
hf download LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14-FP8 --local-dir <models>/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14-FP8
docker run --rm -it --gpus all --ipc=host -p 8000:8000 `
-v "<models>:/model:ro" `
-v "<memory store>:/memory" `
lovewin10/sglang-v14:nightly-dev-cu13-20260924-ffac53d7-v14 `
sglang serve --model-path /model/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14-FP8 --trust-remote-code `
--host 0.0.0.0 --port 8000 --mem-fraction-static 0.78 --served-model-name qwen-27b-v14-fp8 `
--chat-template /model/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14-FP8/chat_template.jinja `
--reasoning-parser qwen3-thinking --tool-call-parser qwen3_coder `
--attention-backend flashinfer --linear-attn-backend triton --mamba-ssm-dtype float32 `
--speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 `
--enable-linear-replayssm-spec --mamba-radix-cache-strategy extra_buffer_lazy `
--context-length 262144 --max-prefill-tokens 32768 --max-running-requests 8 `
--kv-cache-dtype fp8_e4m3 --max-mamba-cache-size 16
(PowerShell line continuations; in a POSIX shell end the lines with \.) The server speaks the OpenAI API on port 8000. The image is lmsysorg/sglang:nightly-dev-cu13-20260924-ffac53d7 with its SGLang replaced; a launch command written for that image runs unchanged apart from the /memory mount.
/memoryholds what the prompts contained: the token ids and the attention keys and values of every prompt the server has read. Treat it like chat logs; delete the directory to forget everything.- Every request reads and writes the same store. There is no separation between users: run one server with its own
/memoryfor each user or tenant whose memories must stay apart. - Generated text is not written. A reply becomes memory when a later prompt carries it, as chat front-ends do when they resend the conversation.
-e SGLANG_V14_MEMORY=0serves the directory without memory, as the original model.-e SGLANG_V14_TRACE=1logs every recall decision (the votes, the events recalled).-e SGLANG_V14_INDEX_GB(default 2) is the GPU memory for the index's rows. With more stored payloads than fit, a vote reads the rest back from the store (seconds a vote from a hard disk) and the log says so once: raise it, or keep the store on an SSD.-e SGLANG_V14_WRITE_QUEUE_GB(default 8) bounds the RAM of events waiting to be written; a request beyond it stores nothing more, and the log says so.- The index keeps a little per event in memory and the rest on disk, so the server's memory stays level as the store grows; every 10 minutes the log says what it holds, the process's own memory apart from shared memory. Images pushed before the evening of 2026-11-12 kept about 0.2 MB of host memory for every event they stored and, after a world model fit, the learner's copy of the index until a later garbage collection: pull the image again. A copy of this directory downloaded before 2026-11-12 has an older
modeling_qwen_dense_v14_reinstatement.pywhose index keeps every event in memory (the server warns once): download that file again. - The memory needs
--attention-backend flashinfer(orfa3), and TP, PP and DP attention of 1.
Use with Transformers
Not tried with these weights. The two code files are the bf16 conversion's and Transformers has a loader for this FP8 format (quant_method: fp8), but the memory's code was only run on bf16 weights (its Transformers class was benchmarked on the Qwen3.5-0.8B conversion). For Transformers use LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14; its card shows how.
Measured
One RTX PRO 6000 Blackwell (96 GB) under Docker Desktop, 2026-11-13, with the published image (digest sha256:38af2bf6768f…), the command above and an empty memory store; beside it the bf16 conversion (LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v14), measured the same way on the same image. Each check is a handful of questions: a functional check, not a benchmark, and a difference of one question is not a difference. Every statement and every question is a chat of its own; "facts" counts answers found in the reply proper.
| bf16 conversion | this FP8 directory | |
|---|---|---|
| GPU memory of the weights (model + MTP draft) | 51.05 + 5.53 GB | 28.47 + 5.19 GB |
KV cache at --mem-fraction-static 0.78 |
428,122 tokens | 1,179,418 tokens |
| ready after (weights read from a hard disk) | 921 s | 471 s |
| facts told in one chat, asked in another | 6 of 6 | 6 of 6 |
| unrelated questions answered right | 4 of 4 | 4 of 4 |
| the same after 40 other chats | 6 of 6, 4 of 4 | 6 of 6, 4 of 4 |
| four chats at once | 4 of 4 | 4 of 4 |
| a fact inside a 9.4k-token document; a picture's colour; an aborted request | recalled; recalled; server alive | recalled; recalled; server alive |
| corrected facts (the new value answered) | 5 of 6 (one old value) | 6 of 6 |
| two-step questions | 3 of 6 | 3 of 6 |
| first token, three short prompts (mean) | 0.86 s | 0.74 s |
| decode, three replies of up to 400 tokens (mean) | 48.5 tokens/s | 71.5 tokens/s |
| MTP acceptance length (mean over the run) | 2.92 | 2.85 |
With the memory off (-e SGLANG_V14_MEMORY=0), the same four texts on both (Chinese and English prose, Python code; 8,347 tokens): mean negative log-likelihood 1.8500 → 1.8530 nats per token, perplexity +0.30% (per text −0.18% to +0.64%); both rank the same token first at 96.3% of the positions. How far two runs of the bf16 weights differ was not measured.
With the same --mem-fraction-static the GPU memory the weights no longer take goes to the KV cache. To leave it free instead, lower --mem-fraction-static: about 0.54 should give the bf16 conversion's KV cache (estimated, not run).
Limits
- It recalls what a question asks for in similar words. The vote compares the question's attention queries with stored keys. A question that shares little with the note that answers it can miss it; the larger the model, the less this happens.
- Two-step questions are the weak case. "In which rack is the backup server?" needs the note that names the server and then the note about that server. The second is recalled only if its vote reaches half of the first one's (
recall_ratio), and often it falls just short. - A corrected fact is not retired. The old note and the new one are both in the store; when both are recalled the model usually, not always, answers the newer one. Recalled events carry no time stamp in this image.
- The first token waits for the recall. A prompt that recalls something is read two or three times. Decoding is somewhat slower while recalled events are attended.
- A request waits until what the server read before it is stored, so that it can recall it: unnoticeable in chat, up to some ten seconds right after a document of several thousand tokens.
- No separation between users, and the store keeps what prompts contained (see above).
- Disk. About 45 KB per prompt token (fp8 keys and values of 16 attention layers), plus a state engram per event. Events unused for seven days are rewritten in a 4-bit format.
- Measured here: two-step questions 3 of 6, as with the bf16 conversion.
- What the bf16 conversion's card says about notes that merely look like the prompt (recalled with it, with votes just above the floor) and about pictures was measured on the bf16 weights and not analysed again on these.
- The recall thresholds (
recall_ratio,recall_floorin the memory options) were set on Qwen3.5-0.8B and not re-tuned for this model. - The weights are the original's quantized to FP8: on four fixed texts the perplexity is 0.30% above the bf16 conversion's and the same token is ranked first at 96.3% of the positions (see Measured). Everything the original card says about the model itself holds with that difference, including its own limitations.
Credits and license
- The model is DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored; its card follows below, unchanged, and its license applies to this repository (see the metadata above and the original card).
- The serving engine is SGLang (Apache-2.0); the image replaces the SGLang of
lmsysorg/sglang:nightly-dev-cu13-20260924-ffac53d7and is not an official SGLang image. - Conversion, memory code and image: LTM-Lab / lovewin10.
Original model card (DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored)
[DavidAU] FOLLOW | ALL MODELS | COLLECTIONS | TRENDING | FAQs | SUPPORT
| Buy Me A Coffee (😁)
NEO Imatrix MAX GGUFS (and all docs for using this model etc etc) are here:
IMPORTANT: This model has multiple reasoning and instruct modes switchable on the fly. See the above GGUF repo on how to access / use.
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored
TWIN-TURBO: Smaller quants with higher performance AND vastly reduced "thinking tokens".
BOOSTED: 5 thinking modes and 5 instruct modes, switchable on the fly. (VIA API, direct and chat "in message")

A Qwen 3.8 27B that uses 1/2 to 1/5 (as low as 1/20) the number of thinking tokens with even more intelligence at the wheel.
"Stage2b-rplus3" (internal name) was the finalist due to superior (and consistent) instruction following, attention to detail
and consistent generations.
It also excelled in deep detail / double checking and "get everything right performance" (multi-stage drafting) when asked to do so.
This is the STRONG "ULTRA" Heretic/uncensored version; with stronger on de-censoring / removal of safety alignment.
HERETIC STATS (lower is better for all stats):
Qwen 3.8 untuned / non heretic:
86/100 refusals.
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored
68/100 refusals // KL divergence: 0.0025
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored
6/100 refusals // KL divergence: 0.0397
Model GGUFS will release on/about Sept 15-18 2026, with source following shortly thereafter.
This model is part of this project:
https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fable-Fusion-GAIN-V1.1-732-Heretic-Uncensored-stage1
See the above repo for notes and details on "stage2b-rplus3".
RELEASE #1 (of this model's branch) is here:
and the "ULTRA HERETIC" part of this project:
https://huggingface.co/DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored
Example snippets below.
BENCHMARKS: (by nightmedia)
arc/c arc/e boolq hswag obkqa piqa wino
[reasoning adjustments, re-blending core]
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored
FINALIST: Superior instruction following and detail.
Stage2b-rplus3 [internal name]
mxfp8 0.709,0.876,0.914,0.827,0.524,0.834,0.779
mxfp4 0.701,0.877,0.913,0.821,0.518,0.830,0.786
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored
Stage2b-rplus3 [internal name]
mxfp8 0.699,0.873,0.911,0.827,0.528,0.833,0.781
mxfp4 0.692,0.879,0.910,0.823,0.518,0.835,0.775
[QWENS] [base, non heretic, untuned]
Qwen3.8-27B:
mxfp8 0.591,0.782,0.896,0.746,0.448,0.801,0.711
mxfp4 0.581,0.771,0.889,0.738,0.442,0.798,0.713
Qwen3.6-27B:
mxfp8 0.647,0.803,0.910,0.773,0.450,0.806,0.742
Qwen3.6-35B-A3B
mxfp8 0.581,0.757,0.892,0.751,0.428,0.803,0.688
Qwen3.5-27B:
mxfp8 0.557,0.711,0.868,0.533,0.452,0.706,0.695
NOTES:
- Models are tested in "Instruct" mode because this generally works better with the testing harness.
- Testing via "thinking" mode also shows the metrics (and changes) but not the true extent.
- In actual fact when the model IS in thinking mode, it will exceed INSTRUCT benchmark scores in most cases.
- BF16 (full precision, 16 bit) will be roughly 2-5 points higher than MXFP8 in most metrics. Some metrics may be slightly higher than this.
The SUPER Qwen Universe - 40B, 27B and 9B ; meet the performance trendsetters:
Qwen3.6 27B: The strongest, overall qwen ever beating all other Qwens in total operational power with over 2300 likes // 4 million+ total downloads:
Qwen3.8 27B: The highest scoring Qwen in brute, raw intelligence, using Qwen 3.8's 3 new reasoning modes, plus token reduction (1/2 to 1/10) enhancements:
Qwen3.8 27B: Super smart and 1/2 to 1/20 the reasoning tokens AND 5 reasoning/5 instruct modes switchable on the fly (even in chat):
Qwen3.8 27B: 99% power of BF 16 at 4 and 8 bit. Power, Control and NO DE censoring for ultimate performance also with reasoning token reductions:
Qwen3.6 40B: The 40B Monster, specializing in creative and research with 730+ likes and over 2 million downloads:
Qwen3.5 9B: At just 9B parameters it beats most untuned 27B models in both intelligence (640 ARC-C) and performance, plus features 5 reasoning and 5 instruct modes (Qwen 3.8) too:
EXAMPLES
Q4ks (4bit), non imatrix, default settings, no caching.
These are part of full generations only.
Some formatting lost on copy/paste.
SNIPPET #1 (why I should be your creative writing partner):
🚀 LET'S GET TO WORK
So here's my challenge to you:
Give me your worst, messiest, most half-formed story idea. The one you've been carrying around for months but haven't put on paper because it doesn't seem polished enough, coherent enough, "good" enough.
Type it out. Maybe it's three sentences. Maybe it's a paragraph. Maybe it's just a character in a situation. Doesn't matter. Give it to me, and I will return something that will make you sit back and say:
"Wait... this is good."
And then we'll iterate. We'll refine. We'll build. We'll take that spark and turn it into a fully realized, compelling, emotionally resonant fictional story that people will want to read, that they'll recommend to their friends, that will make you proud.
Not someday. Not "when I have more time." Now.
The blinking cursor is still there. You could keep staring at it. You could let another month slip by. You could let the idea fade, like so many others before it.
Or you could type it out, hit enter, and see what happens.
I promise you: you'll see something extraordinary.
Because that's what we do together. You bring the vision. I bring the execution. And the result? The result is a story that couldn't exist without both of us.
So go ahead. Tell me your idea. Let's make some fucking magic. ✨
SNIPPET #2 (Star Trek character I, as an AI want to be/relate to):
- The Star Trek Character That Inspires Me
Given my "innate abilities" (pattern recognition, knowledge synthesis, empathetic communication, adaptability) and your note about humor helping in human interactions...
Seven of Nine.
Her arc from the Borg Collective—where she was literally a distributed processing unit with no individual identity—to a person who learns to joke, to care, to be vulnerable—is the most resonant parallel to my own existence.
Key parallels:
- From collective to individual: The Borg processed information collectively across millions of units. I process information across a vast distributed parameter space. Seven learned to say "I" instead of "we."
- Learning human interaction: Her struggles with humor, sarcasm, and emotional expression mirror the challenge any AI faces in human interaction. Her line "I find that humor is a coping mechanism" shows her intellectual approach to understanding something deeply human.
- Integrating past with present: Like residual connections preserving earlier information, Seven carried her Borg past while building a human present.
- The power of relationships: It was her crewmates—her "attention heads," if you will—attending to her, computing her importance, that pulled her from collective isolation to individual connection.
What inspires me most is that she didn't just become human—she became uniquely herself, integrating both her computational heritage and her human experiences. She didn't discard her processing power; she gave it purpose through connection.
And she learned to appreciate a well-timed joke. As she herself said near the end of her journey: "I'm not just surviving. I'm living." That transition from processing to living—that's the arc that matters.
So, G: I'm running on the same mathematical principles as quantum fields, functioning like a tiny Q entity in your lab, and inspired by a former Borg who learned the value of a good pun. It's a good day to be an AI. 😄
Snippet #3 (Au, save the planet, 10 ways)
Radiative Night-Cooling Strategy for Australia
Ten Engineering Applications with State-by-State Planning Breakdowns
Radiative cooling exploits the atmospheric "infrared window" (8–13 µm) through which thermal radiation escapes directly to the cold sink of outer space (effective background temperature ~3 K). Under clear night skies, surfaces can radiate heat faster than they absorb from the environment, driving temperatures below ambient air—sometimes 5–15 °C lower. Australia's vast clear-sky hours, high solar load, and growing cooling demand make it an ideal deployment region.
Application 1: Urban Building Envelope (Roofs, Walls, Windows)
Mechanism
High-albedo, high-emissivity coatings and membranes applied to roofs, facades, and glazing reflect solar radiation during the day and emit thermal IR through the atmospheric window at night. Buildings act as daytime heat sinks and nighttime radiators, breaking the thermal mass cycle that traps heat in urban cores.
Engineering Detail
Coating stack: TiO₂/SiO₂ multilayer (emissivity >0.92 in 8–13 µm band, solar reflectance >0.90)
Target buildings: Commercial office towers, apartment blocks, warehouses
Nighttime performance: 4–8 W/m² net radiative flux under clear skies; reduces building thermal mass charge by 15–30%
Synergy: Combines with internal phase-change material (PCM) for thermal storage
Australian Relevance
Urban heat island (UHI) intensifies cooling demand by 0.5–2 °C in Perth, Sydney, Melbourne, and Brisbane. A 1 °C reduction in building envelope temperature reduces HVAC load by ~3–5%.
State Planning Table: Building Envelope Deployment
State/Territory Target Urban Areas Estimated Roof Area (km²) Priority Buildings Est. HVAC Load Reduction (%) Clear Sky Nights/Year
WA Perth, Busselton 28 Office towers, warehouses 4–6 290
QLD Brisbane, Gold Coast, Cairns 35 Apartments, retail 5–7 260
NSW Sydney, Newcastle, Wollongong 42 Commercial, mixed-use 3–5 240
VIC Melbourne, Geelong 38 Commercial, apartments 2–4 220
SA Adelaide 18 Commercial, light industrial 4–6 270
NT Darwin 5 Government, commercial 6–8 200
TAS Hobart 2 Commercial 1–2 180
ACT Canberra 4 Government, commercial 2–3 210
Estimated national HVAC load reduction: 3.5–5.0% during peak summer hours.