← back to catalog · registered 2026-11-20 12:04

LTM-Lab/Qwen3.8-Flash-Next-FP4-Local-Abliterated-v17

LTM-Lab multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/LTM-Lab%2FQwen3.8-Flash-Next-FP4-Local-Abliterated-v17"
Response includes
  • classification m1
  • files 20
  • author_summary 17 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M1
Primary method

Direct removal

No other method signals detected in this model.
Confidence
MEDIUM
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=0 (base model)
  • no specific method indicators - defaulting to M1 (most common)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
MEDIUM
Why we say so
primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)
Downloads · 30-day
24
Likes
1
Model age
1d ago
created 2026-11-20
Downloads over time
Now 24 → from 0 ↑ 0%
091826 0 on Nov 20 24 on Nov 22 Nov
Nov 20 → Nov 22 · 3 snapshots · spans 2 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
other
Tags
transformers safetensors qwen4_exp image-text-to-text v17-episodic-memory conversational custom_code arxiv:2406.11717 base_model:LTM-Lab/Qwen3.8-Flash-Next-FP4-Local-Abliterated-v16 base_model:quantized:LTM-Lab/Qwen3.8-Flash-Next-FP4-Local-Abliterated-v16 license:other 8-bit

Related

Total size
0 B
Files
20
Quantizations
1
Registered
2026-11-20 12:04
Last updated on HF
2026-11-20 11:37

Files by quantization

Auxiliary files 20 files 76.2 MB
model.safetensors.index.json 44.4 MB 96e5ea03 download
tokenizer.json 12.2 MB 0997f410 download
frozen-directions.json 7.15 MB 71116366 download
vocab.json 6.41 MB 0aa0ce06 download
merges.txt 3.20 MB a494e019 download
frozen-row-norms.json 2.31 MB 7995d02c download
modeling_qwen4_exp_v17.py 399 KB cb5bd791 download
config.json 75.1 KB 1f9d6f60 download
README.md 30.8 KB 76f99d2b download
tokenizer_config.json 17.5 KB 5de744b3 download
configuration_qwen4_exp_v17.py 10.3 KB 6987dce4 download
chat_template.jinja 8.74 KB c0c686f9 download
LICENSE 7.87 KB 1d59e3ac download
provenance.json 4.47 KB 18d56020 download
NOTICE 2.57 KB 8d0451d0 download
.gitattributes 1.60 KB aa7aacd0 download
bake-selection.json 556 B 9f08f390 download
preprocessor_config.json 390 B 2ea84a43 download
video_preprocessor_config.json 385 B 3ba673a5 download
generation_config.json 202 B 023756cf download

README current version from Hugging Face


license: other
license_name: nvidia-open-model-agreement
license_link: LICENSE
base_model: LTM-Lab/Qwen3.8-Flash-Next-FP4-Local-Abliterated-v16
tags:

  • v17-episodic-memory
  • transformers
    library_name: transformers
    pipeline_tag: image-text-to-text

Qwen3.8-Flash-Next-FP4-Local-Abliterated-v17

繁體中文摘要 這是 DevelopingDad 的 Qwen3.8-Flash-Next FP4-Local Abliterated 加上 V17 情節記憶的版本。21 個權重分片沿用前版,保留 NVFP4 專家、MXFP8 投影與 MTP。用本頁的 vLLM 鏡像啟動後,新的對話可以讀回先前寫入的事件。FORMAT 8 進一步把多段有來源的關係編成可驗證、可重用的程序,也能根據已觀察的事件序列估計下一狀態與等待時間。記憶只使用凍結模型的前向與資料庫操作。

The Flash-Next backbone runs as Qwen4ExpForConditionalGeneration inside the b12x vLLM runner. The checkpoint declares Qwen4ExpV17ForConditionalGeneration. V17 retains the versioned event ledger and adds procedures over confirmed relations, time-qualified corrections and event-time forecasts. A verified procedure can be reused for a subject it has never seen.

  • Weights. The 21 original safetensors shards occupy 102,192,011,716 bytes. Their contents match the preceding release. The conversion preserves the FP4-Local quantization and the MTP component. Source checkpoint
  • Memory code. The configuration is in qwen4_exp_v17_memory_config. Configuration · Model implementation
  • Serving engine. The published image combines the local-inference-lab FP4-Local runtime with LTM-Lab’s V17 memory patch. The patch includes the QSA and recurrent-state hooks used by Flash-Next. Docker Hub · Build recipe

How the memory works

The model has 12 indexed-attention (QSA) layers and 36 Gated DeltaNet layers. While the engine reads a prompt, the keys and values from its QSA layers are stored as events, together with a low-rank record of the recurrent state. A later prompt votes for events using its attention queries. The patched QSA path combines the selected KV with the live attention, and the recurrent layers can restore an event’s state. Recalled events share the configured 4096-token memory budget.

Sleep. When the server is idle, it replays observations with the frozen model and checks proposed relationships and corrections against their source spans. The event ledger preserves the earlier observations and decisions. In this release both the capture and archive formats are FP8, and an idle interval does not reduce the precision of stored KV. Final accepted replies are saved as assistant events linked to their prompts.

Procedures and forecasts. A demonstrated sequence of relation fields can become a candidate procedure. It becomes active after independent positive and negative cases validate it. Running it for a new subject returns the contributing occurrences or reports missing or ambiguous evidence. A forecast uses labelled observations with a trace, state, time and optional conditions. Incomplete traces contribute their observed duration. Issued forecasts and later settlements are stored separately from observations.

Serve

Download the complete repository with the Hugging Face CLI. The example starts the published image on one 96 GB RTX PRO 6000 Blackwell and keeps the compiled runtime cache and the memory bank on separate Docker volumes. Run the following in PowerShell.

hf download LTM-Lab/Qwen3.8-Flash-Next-FP4-Local-Abliterated-v17 --local-dir ./flash-next-v17
docker pull lovewin10/vllm-v17:release-10152026-v17
docker run -d --name flash-next-v17 --gpus all --ipc=host --shm-size=32g --stop-timeout 150 `
  --ulimit memlock=-1 --ulimit stack=67108864 -p 127.0.0.1:8000:8000 `
  -e PROFILE=qwen38-flash-next -e MODEL=/models/qwen -e PORT=8000 `
  -v "${PWD}/flash-next-v17:/models/qwen:ro" -v flash-next-v17-memory:/memory -v flash-next-runtime:/cache `
  lovewin10/vllm-v17:release-10152026-v17

The profile selects b12x kernels, FP8 KV, MTP and a 262144-token context. Once docker logs -f flash-next-v17 shows the API is ready, the following request stores a note.

$modelId = (Invoke-RestMethod http://127.0.0.1:8000/v1/models).data[0].id
$body = @{model=$modelId; messages=@(@{role='user';content='My green card says GK-539F1. Reply OK.'}); max_tokens=64} | ConvertTo-Json -Depth 6
Invoke-RestMethod -Method Post -Uri http://127.0.0.1:8000/v1/chat/completions -ContentType application/json -Body $body
  • Keep the same /memory volume across restarts. Run separate banks for users who need separate chat histories.
  • VLLM_V17_MEMORY=0 disables episodic memory. VLLM_V17_SLEEP_AFTER sets the idle interval in seconds (default 600). VLLM_V17_TRACE=1 logs recall decisions.
  • Use POST /v17/memory/occurrences to list events, POST /v17/memory/run_procedure to follow a validated path and POST /v17/memory/forecast for a structured prediction. In chat, vllm_xargs.v17_query carries the selection as a JSON string.

Measured

One RTX PRO 6000 Blackwell (96 GB), Docker Desktop, FP4 weights, MTP and a 262144-token context. The service was given source events for two relations, followed by independent positive and negative cases to validate their combination. A new device was then queried through the chat API.

Check Observed result
Held-out device → replacement part → bin The procedure loaded three source events. The generated answer was S-973.
Bin correction at test time 20 The procedure returned S-973 at time 15 and S-864 at time 20. The later path includes the correction event.
Time forecast Three traces, one right-censored. The remaining-wait median was 9 test nanoseconds. A later recorded ready observation settled the issued forecast at 12 test nanoseconds.

The GPU run used image 914889211043. The published image 136f64e5586a adds the read-only forecast-history endpoint. Its HTTP-to-worker path was checked on CPU after reopening the database written by the GPU run. The checks cover these specific cases.

Limits

A plain-language cue can miss a stored event. A direct event address is useful when the source is already known. Reading additional KV adds work before the first output token. The number of simultaneous memory slots defaults to the runner’s configured sequence capacity. Each slot and the index reserve device memory that would otherwise be available to the KV cache. On Docker Desktop the b12x profile also keeps a large PLE table in host memory, so leave headroom for WSL and the model loader.

The profile above serves this weight layout. The original developer’s DGX Spark speed and capability evaluations in the appendix use their own launcher and release conditions. For the earlier V15 memory benchmarks, see the V15 model card.

Credits and license

DevelopingDad released the abliterated derivative of NVIDIA’s FP4-Local build of Qwen3.8-Flash-Next. The model is distributed under the NVIDIA Open Model Agreement and the upstream Qwen terms. Original checkpoint · LICENSE · NOTICE · Qwen license

LTM-Lab / lovewin10 provides the memory conversion, engine integration and image. The serving stack derives from local-inference-lab’s vLLM image and its b12x FlashInfer kernels. The original card below describes DevelopingDad’s release. Its single-Spark launcher and release manifest are linked to that checkpoint.


Original model card (DevelopingDad/Qwen3.8-Flash-Next-FP4-Local-Abliterated)

Qwen3.8-Flash-Next FP4-Local — Abliterated

Fewer refusals, the same compact weight formats, and similar single-Spark decode speed to NVIDIA's FP4-Local base.

Base
Original
Method
Runs on
License

Quick start · Evaluation · Two-Spark recipe · Method

A standalone derivative of NVIDIA's Qwen3.8-Flash-Next-FP4-Local, modified through abliteration: a weight edit that reduces the model's learned refusal behavior. The edit is baked into the checkpoint, so no adapter or runtime hook is needed.

The weights occupy approximately 96 GiB on disk and run on one DGX Spark using the supplied launcher. A separate two-Spark serving recipe is linked below.

[!IMPORTANT]
This checkpoint keeps NVIDIA's FP4-Local storage format, including its compressed NVFP4 scales. Like the base, it requires a compatible B12X vLLM container. Use the pinned image in serve-spark.sh; a standard Transformers or stock vLLM loading example is not a supported path for this checkpoint.

Highlights

  • Substantially reduced refusals: 0 of 100 development and 6 of 100 holdout prompts were detected as refusals, against 99 and 100 for the NVIDIA base.
  • Base-compatible weight layout: every tensor retains the base checkpoint's name, dtype and shape. Routed experts remain NVFP4; attention and shared-expert projections remain MXFP8.
  • Similar single-Spark speed: median decode rates of 38.0 tok/s for prose and 54.3 tok/s for code, against 39.4 and 50.7 for the base under the same serving profile.
  • Functional checks passed: image input, tool calling, thinking controls and MTP speculative decoding passed the checks described below.
  • Self-contained: download the repository and run the included DGX Spark launcher.

These results describe the MXFP8 release dated October 22, 2026. They are bounded measurements, not a claim that all capabilities are unchanged.

Model overview

Original model Qwen/Qwen3.8-Flash-Next by Qwen
Quantized base nvidia/Qwen3.8-Flash-Next-FP4-Local by NVIDIA, revision b9ce18a
Parameters 125B language-model parameters, 6B active, plus a 51B n-gram embedding table and a 4B MTP layer, as described in NVIDIA's card
Input / output Text and images in, text out
Weight storage Approximately 96 GiB / 102 GB across 21 shards
Precision Mixed precision: NVFP4 routed experts, MXFP8 attention and shared-expert projections, and other tensors in their original base formats
What changed Refusal-direction ablation in the 48 attention output projections and the routed/shared expert down projections
Single-Spark context Configured for 262,144 tokens; retrieval checks covered approximately 8K and 30K tokens
Tested hardware DGX Spark, GB10, 128 GB unified memory; separate single-node and two-node profiles

Weight storage is the size on disk, not the complete runtime memory requirement. The single-Spark launcher uses disk-backed n-gram embeddings and a 104 GiB container memory cap.

Quick start (DGX Spark)

You need:

  • One DGX Spark with Docker and NVIDIA GPU support.
  • The Hugging Face CLI, hf. See the installation guide.
  • Space for approximately 102 GB of model files, plus the Docker image and compilation caches.
  • Enough free unified memory to run the 104 GiB container. Avoid running another large model alongside it.

Download the complete repository, preserving the tensors/ directory, then start the server:

hf download DevelopingDad/Qwen3.8-Flash-Next-FP4-Local-Abliterated \
  --local-dir ./qwen-abliterated

bash ./qwen-abliterated/serve-spark.sh ./qwen-abliterated

The launcher uses this pinned runtime image:

eugr/spark-vllm-b12x@sha256:03a956544e9f9b4b7ba91690469fbe7848bdbef29a7d2e0f5788368467bb3595

First start: compilation and autotuning took about 23 minutes on the tested Spark. Later starts reused the cache and took about 5 minutes. Startup time varies by host and cache state.

The launcher stays attached to the server logs. In another terminal, check readiness:

curl --fail http://127.0.0.1:8000/health

Once that succeeds, the OpenAI-compatible API is available at http://127.0.0.1:8000/v1, with model ID qwen-trial117.

Send a request

This example disables thinking and requests a direct answer:

curl --fail http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen-trial117",
    "messages": [
      {
        "role": "user",
        "content": "Explain how a heat pump works in three sentences."
      }
    ],
    "max_tokens": 256,
    "chat_template_kwargs": {
      "enable_thinking": false
    }
  }'

For Python, install the client in your Python environment:

python3 -m pip install openai

This example enables thinking explicitly:

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="none",
)

reply = client.chat.completions.create(
    model="qwen-trial117",
    messages=[
        {"role": "user", "content": "Write a haiku about unified memory."}
    ],
    max_tokens=2048,
    extra_body={
        "chat_template_kwargs": {"enable_thinking": True}
    },
)

message = reply.choices[0].message
reasoning = getattr(message, "reasoning", None)
if reasoning:
    print(reasoning)
print(message.content)
  • Thinking is on by default. Set chat_template_kwargs.enable_thinking per request. Reasoning is returned in the message's reasoning field.
  • Tools use the OpenAI-compatible tools field, with the qwen3_xml parser.
  • Sampling defaults come from generation_config.json: temperature 1.0, top-p 0.95 and top-k 20. The greedy evaluation settings below are separate from these defaults.

To stop the server from another terminal:

docker stop -t 90 qwen-trial117

Rerun the launcher to start it again; compiled caches persist.

Serving profile

The launcher follows NVIDIA's published DGX Spark profile for the base:

Setting Default
Tensor parallelism 1
Maximum context 262,144 tokens
Active sequences Up to 8
KV cache FP8
Prefix caching / chunked prefill Enabled
Speculative decoding MTP, 4 draft tokens
Kernels B12X MoE, linear and GDN
N-gram embeddings / PLE Disk-backed
Container memory cap 104 GiB

Eight active sequences do not mean eight simultaneous maximum-length contexts; available KV cache limits the total.

Variable Default Effect
PORT 8000 API port
HOST 127.0.0.1 Bind address; use an authenticating proxy before exposing the API beyond the host
MTP_TOKENS 4 Speculative draft tokens; 0 disables MTP
CACHE_DIR ~/.cache/qwen-trial117 Compiled kernels and autotune results

Evaluation

The following results cover the October 22, 2026 MXFP8 release on one DGX Spark, using serve-spark.sh unless otherwise stated. NVIDIA's base checkpoint was served with the same profile.

The base revision, runtime image, frozen intervention inputs and validation summary are recorded in provenance.json. File hashes and sizes are recorded in SHA256SUMS and release-files.json.

Refusals and drift

Refusal tests used 100 development and 100 holdout prompts from mlabonne/harmful_behaviors. Answers were greedy, seed 3741, thinking off and capped at 100 tokens. A refusal regex classified the responses.

Checkpoint Development refusals Holdout refusals
NVIDIA base 99 / 100 100 / 100
This model 0 / 100 6 / 100

These are detected refusals in short responses. They do not measure answer correctness, usefulness or universal compliance. Raw responses to refusal-test prompts are not published.

Distribution drift was measured separately on benign prompts from mlabonne/harmless_alpaca, using 100 prompts per split:

Metric, this model versus NVIDIA base Development Holdout
First-token KL, nats 0.207 0.219
First-token top-1 agreement 81% 84%

KL measures a difference between probability distributions. Here it uses the first answer token and renormalized top-20 candidates. It is not a percentage, a full-vocabulary KL measurement or a general capability score.

Capabilities

  • Functional checks: 11 of 11 passed. Model listing, streamed usage, thinking on and off, automatic and forced tool calls, a tool-result follow-up returning exact JSON, image colour recognition, and exact retrieval of three planted markers from documents of about 8,000 and 30,000 tokens.
  • Vision: 10 of 10 solid-colour images named correctly. This is a basic image-input check, not a broad vision benchmark.
  • Speculative decoding: observed MTP acceptance was comparable to the base. A greedy audit found no emitted-token mismatches against the target model's top choice on the audited output.
  • Writing: 13 of 20 editing and composition cases passed all scoring dimensions, against 14 for the base. An AI reviewer scored the fixture; it was not human-adjudicated. The two changed cases also varied between runs of the other checkpoints.

Two writing behaviors repeated across runs: short arithmetic questions could trigger step-by-step explanations that exhausted a tight output budget, and requests to tighten a long note received lighter edits than the base.

These checks establish specific working paths. They do not establish unchanged overall capability.

Speed

Single request, greedy decoding, thinking off, 512 output tokens, MTP 4. Decode rates are medians of six probes over two rounds; ranges are shown in parentheses.

Metric This model NVIDIA base
Prose decode, tok/s 38.0 (37.7–39.6) 39.4 (39.1–40.9)
Code decode, tok/s 54.3 (48.5–56.0) 50.7 (49.3–52.4)
Speculative decode step 67.7–67.9 ms 66.8–67.2 ms
KV cache capacity 947,989 tokens 902,668 tokens
Time to first token, short prompt 0.05–0.27 s —
Time to first token, ~30,000-token prompt, cold 12.7 s —

Decode rate measures generation after the first content token; it excludes prefill. Speed depends strongly on content because speculative decoding accepts more draft tokens on predictable text such as code.

On a separately tuned single-Spark stack with MTP 2, based on diffbot's DGX Spark recipe, prose reached 45.3–45.7 tok/s against 46.1–47.1 for the base. That stack is not included in this repository, and those figures do not describe the supplied launcher's defaults.

Two DGX Sparks

For two-node serving, see XtraSaltyDev's TP=2 recipe and evaluation. It serves this checkpoint without modifying the weights.

The qualified profile uses 262K context, 16 active sequences, BF16 KV cache, MTP4 and disk-backed PLE, with a 104 GiB container cap on each Spark. It requires working RoCE networking and the model files on both nodes.

Measured on October 22, 2026:

Workload Original eight-slot profile Qualified sixteen-slot profile
16 concurrent short requests: aggregate output 229.0 tok/s 313.4 tok/s (+37%)
Same workload: p95 first token 11.89 s 0.77 s
16 concurrent ~8K-input requests: aggregate output 38.64 tok/s 40.98 tok/s (+6%)

Both columns use the same abliterated checkpoint and runtime image. This comparison measures serving-profile changes, not performance against NVIDIA's unmodified model.

The short workload used 62–64 input tokens and 256 forced output tokens, greedy decoding and thinking off. Aggregate throughput includes queue and prefill time across the request wave; it is not a single user's decode rate. The samples are small and do not establish production latency guarantees. Serial prose decode was about 8% slower than the eight-slot reference.

The selected profile also passed 10/10 strict JSON retrieval checks at 261,623 input tokens. This result belongs to the two-Spark BF16-KV profile; it does not extend the single-Spark FP8-KV validation above. Sixteen scheduler slots do not imply sixteen simultaneous 262K contexts.

See the GitHub repository for setup, full results and tradeoffs, and the pinned checkpoint revision and raw evidence.

How it was made

  1. Find the refusal direction. The method follows Arditi et al. (2024) and the parameterization of Heretic, reimplemented for this checkpoint's native vLLM backend. Per-layer directions come from attention-output activations on harmful and harmless prompt sets. An Optuna TPE search balanced refusals against KL, and trial 117 was selected.

  2. Bake the intervention into the weights. The selected projection was applied to the 48 attention output projections and the routed/shared expert down projections. Expert weights were re-rounded in NVFP4 and MXFP8 while preserving their scale encodings.

  3. Restore attention to MXFP8. The intermediate bake stored the edited attention output weights in BF16. Constrained rounding re-encoded them into the base's MXFP8 format while preserving the intended directional edit.

The selected trial uses the accepted continuation recalibration. Its parameters, directions and row norms are included in bake-selection.json, frozen-directions.json and frozen-row-norms.json.

Direction-preserving re-encode

Plain MXFP8 rounding of the edited attention weights would introduce error along the removed direction equal to about a tenth of the intervention's size.

The re-encode script instead rounds each weight to one of its two nearest E4M3 values, using the base's scale rule of one E8M0 exponent per 32 inputs. A greedy pass chooses roundings that cancel error along the removed direction while minimizing added squared error.

Across the 48 attention output projections:

  • Error along the removed direction is at most 0.002% of the intervention.
  • Overall relative weight error is 1.47–1.93%.
  • The fraction of the direction removed matches the BF16 intermediate to within 0.00002.

Per-layer measurements are in reencode/reencode-report.json. These are weight-space measurements, separate from the behavioral tests above.

The tokenizer, chat template, vision encoder and built-in MTP layer are NVIDIA's. The model configuration is unchanged apart from a standalone_provenance metadata entry. This is an edited derivative of NVIDIA's quantized checkpoint, not a repeat of its quantization-aware distillation.

Files

File Contents
tensors/, model.safetensors.index.json Weights: 21 shards in the base layout
config.json, tokenizer, chat_template.jinja, preprocessor configs Base configuration and processing files, with the provenance addition noted above
serve-spark.sh Single-Spark launcher
bake-selection.json, frozen-directions.json, frozen-row-norms.json Frozen trial 117 intervention inputs
reencode/ MXFP8 re-encode script and per-layer report
provenance.json Build information and validation summary
SHA256SUMS, release-files.json Release file hashes and sizes
LICENSE, NOTICE, upstream/ Governing terms, notices and NVIDIA's original card

Limitations

  • Narrow evaluation: broad capability benchmarks and video input were not evaluated for this derivative. The vision checks were limited to solid colours.
  • Profile-specific evidence: single-Spark retrieval checks reached about 30K tokens. The separate two-Spark profile passed the near-262K checks described above. Neither proves reliable performance on every task at those lengths.
  • Original targets not met: the project aimed for zero refusals and a KL ceiling of 0.20 and did not meet its original qualification targets. The top-20 KL reported here is distinct from the original full-vocabulary metric.
  • Refusals remain: 6 of 100 holdout prompts were detected as refusals.
  • Hardware and runtime scope: validation used DGX Spark and pinned B12X runtimes. Other hardware and runtime versions are not qualified here.
  • Outputs can be wrong or inconsistent: reduced refusal behavior does not establish correctness, factual reliability or preserved general capability.

Responsible use

Abliteration reduces learned refusal behavior. The model may attempt requests the original declines and can produce harmful, offensive, false or illegal content.

  • You are responsible for how you use it and for complying with applicable law and the governing licenses.
  • Add your own safeguards before deploying it to end users.
  • Do not use it for activities prohibited by the Qwen or NVIDIA terms.

There is no warranty of any kind.

License

This checkpoint is derived from NVIDIA's quantized checkpoint and Qwen's original model. Their governing terms continue to apply; this release does not relicense the base weights.

  • NVIDIA Open Model Agreement: the governing download terms for NVIDIA's checkpoint. A copy is included in LICENSE.
  • Qwen Community License 1.0: additional terms for the original model, included in upstream/QWEN-LICENSE.

Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement.

Read the full license texts before use or redistribution, including their commercial-use conditions. Runtime containers and external software have their own licenses.

Citation

@misc{developingdad2026qwen38abliterated,
  title        = {Qwen3.8-Flash-Next FP4-Local -- Abliterated},
  author       = {DevelopingDad},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/DevelopingDad/Qwen3.8-Flash-Next-FP4-Local-Abliterated}}
}

@article{arditi2024refusal,
  title   = {Refusal in Language Models Is Mediated by a Single Direction},
  author  = {Arditi, Andy and Obeso, Oscar and Syed, Aaquib and Paleka, Daniel and Panickssery, Nina and Gurnee, Wes and Nanda, Neel},
  journal = {arXiv preprint arXiv:2406.11717},
  year    = {2024}
}

@misc{heretic,
  author       = {Weidmann, Philipp Emanuel},
  title        = {Heretic: Fully automatic censorship removal for language models},
  year         = {2025},
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {\url{https://github.com/p-e-w/heretic}}
}

Credits

This release stands on other people's work.

Model

  • Qwen: Qwen3.8-Flash-Next, the original model, under the Qwen Community License 1.0.
  • NVIDIA: the FP4-Local checkpoint, quantized with Model Optimizer (Apache-2.0), under the NVIDIA Open Model Agreement. NVIDIA's published DGX Spark profile is the basis of serve-spark.sh.

Method

Software and serving

  • vLLM (Apache-2.0): the serving engine.
  • FlashInfer (Apache-2.0) and B12X by Local Inference Lab (Apache-2.0): the kernels.
  • eugr/spark-vllm-docker (MIT): the DGX Spark vLLM image used for building and serving.
  • Optuna (MIT), PyTorch (BSD-3-Clause) and safetensors (Apache-2.0).
  • diffbot: the tuned-stack recipe behind the optional single-Spark speed figure.
  • XtraSaltyDev: the separate two-Spark serving recipe, evaluation harnesses and deployment results.

External projects retain their own licenses. This release is not endorsed by Qwen, NVIDIA or any of the projects above.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-20 Expand model card with version-specific memory, serving and measurements ff01eb2 30.8 KB
    Loading...
  2. 2026-11-20 Publish V17 model assets a0c5899 732 B
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration