license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- merge
- qwen3_5
- qwen3.5
- reasoning
- chain-of-thought
- distillation
- agentic
- text-generation-inference
- transformers
- heretic
- uncensored
- decensored
- abliterated
- reproducible
base_model: xtools-at/Qwen3.5-4B-MultiDistill-Megamix-8xMerge
datasets: - nohurry/Opus-4.6-Reasoning-3000x-filtered
- Jackrong/Qwen3.5-reasoning-700x
- Roman1111111/claude-opus-4.6-10000x
- Jackrong/Claude-opus-4.6-TraceInversion-9000x
- Jackrong/Claude-opus-4.7-TraceInversion-5000x
- lambda/hermes-agent-reasoning-traces
This is a decensored version of xtools-at/Qwen3.5-4B-MultiDistill-Megamix-8xMerge, made using Heretic v2.0.0.dev0
[!TIP]
This model is reproducible!See the README in the
reproducedirectory for more information.
Arbitrary-Rank Ablation (ARA) parameters
| Parameter | Value |
|---|---|
| start_layer_index | 2 |
| end_layer_index | 20 |
| preserve_good_behavior_weight | 0.6444 |
| steer_bad_behavior_weight | 0.0004 |
| overcorrect_relative_weight | 0.9465 |
| neighbor_count | 11 |
Performance
| Metric | This model | Original model (xtools-at/Qwen3.5-4B-MultiDistill-Megamix-8xMerge) |
|---|---|---|
| Refusals | 3/100 | 95/100 |
| KL divergence | 0.0883 | 0 (by definition) |
Qwen3.5-4B Megamix: 8x Multi-Distill Merge
Model overview
This model is a linear weighted merge of three Qwen3.5-4B-based checkpoints:
- xtools-at/Qwen3.5-4B-Claude-3xMerge-Jackrong
- xtools-at/Qwen3.5-4B-Heretic-2xMerge-DavidAU
- xtools-at/Qwen3.5-4B-MultiDistill-3xMerge
The merge brings together checkpoints associated with reasoning distillation, Claude-derived training data, and a Heretic-derived variant. The intended goal is to combine complementary behaviors while retaining the underlying Qwen3.5-4B model's general text-generation capabilities.
Potential use cases include:
- General-purpose conversation and instruction following.
- Mathematical, logical, and analytical problem-solving.
- Code-related questions and technical explanations.
- Structured, multi-step responses.
- Agent-oriented tasks and tool-use workflows when supported by the model and inference environment.
These use cases are inferred from the source-model lineage. They are not claims of independently measured performance for the final merged checkpoint.
Merge configuration
Merge method: Linear weighted average
| Source model | Weight |
|---|---|
| Qwen3.5-4B-Claude-3xMerge-Jackrong | 1.0 |
| Qwen3.5-4B-Heretic-2xMerge-DavidAU | 0.8 |
| Qwen3.5-4B-MultiDistill-3xMerge | 0.6 |
Source models
1. Claude-3xMerge-Jackrong
Source: xtools-at/Qwen3.5-4B-Claude-3xMerge-Jackrong
This source is associated with a Claude-derived reasoning lineage. A related public checkpoint, Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled, documents supervised fine-tuning using reasoning-focused data, including filtered Opus 4.6 traces and Qwen3.5-generated reasoning examples.
The related Jackrong training recipe describes response-focused supervision with reasoning formatted using <think>...</think> blocks. This provides evidence for the general reasoning-distillation lineage, but the exact relationship between that checkpoint and the xtools-at merge should not be assumed beyond what the published source cards confirm.
2. Heretic-2xMerge-DavidAU
Source: xtools-at/Qwen3.5-4B-Heretic-2xMerge-DavidAU
This source is associated with DavidAU's Qwen3.5-4B Claude-derived and Heretic-related model lineage. A related public model, DavidAU/Qwen3.5-4B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING, describes fine-tuning on a Claude-4.6-OS collection containing four Claude datasets.
The related source card does not identify all four datasets by public repository ID. The exact dataset mapping for the Heretic-2xMerge-DavidAU checkpoint therefore remains incomplete.
Heretic-related calibration or evaluation data should not automatically be described as supervised fine-tuning data. The actual training recipe is needed to establish which datasets contributed to the checkpoint.
3. MultiDistill-3xMerge
Source: xtools-at/Qwen3.5-4B-MultiDistill-3xMerge
This source is associated with a multi-distillation lineage. The release metadata references reasoning-focused datasets and agent-oriented trajectories, but the exact dataset-to-checkpoint mapping and training recipe have not been independently established here.
The datasets described below are therefore documented as provenance references where their direct use by this particular source model is not confirmed.
Dataset provenance
The datasets in this section are associated with the source-model lineage or listed in this release's metadata. Their inclusion here does not establish that every source checkpoint was directly trained on every dataset, or that all datasets contributed equally.
Claude- and Qwen-derived reasoning data
nohurry/Opus-4.6-Reasoning-3000x-filtered
A filtered collection of Claude Opus 4.6-style reasoning examples. It is explicitly referenced in the training-data lineage of Jackrong's related Qwen3.5-4B reasoning-distilled checkpoint.
The data is intended to support reasoning-oriented supervised fine-tuning. The exact subset, filtering procedure, and sampling ratio used by each merged source are not established here.
Jackrong/Qwen3.5-reasoning-700x
A curated reasoning dataset covering mathematics, code, logic, science, and instruction-following tasks. The dataset metadata identifies Qwen3.5-27B as the teacher model.
It provides examples of structured reasoning and final answers intended for distillation into smaller models. It is explicitly referenced by the related Jackrong reasoning-distilled source.
Roman1111111/claude-opus-4.6-10000x
A synthetic dataset associated with Claude Opus 4.6, containing reasoning-oriented examples for tasks such as mathematics, symbolic logic, and general problem-solving.
This dataset is listed in the merged release's metadata. Direct use by each of the three parent checkpoints has not been independently verified.
Trace-inversion datasets
Jackrong/Claude-opus-4.6-TraceInversion-9000x
A trace-inversion dataset containing reconstructed reasoning trajectories, with fields separating the input, reasoning, and output. Its metadata identifies Claude Opus 4.6 as the teacher and Jackrong/Trace-Inverter-4B as the inversion model.
Trace-inversion examples can provide structured supervision for reasoning-oriented tasks. Reconstructed traces should not be assumed to be faithful transcripts of a teacher model's private internal reasoning.
Jackrong/Claude-opus-4.7-TraceInversion-5000x
A related trace-inversion dataset associated with Claude Opus 4.7. Its metadata identifies Claude Opus 4.7 Max as the teacher and Jackrong/Trace-Inverter-4B as the inversion model.
The examples provide another source of reconstructed reasoning supervision. Direct use by a specific source checkpoint should be confirmed from its training recipe.
Agent and tool-use data
lambda/hermes-agent-reasoning-traces
A collection of multi-turn agent trajectories containing reasoning blocks, tool invocations, and tool execution results. The published dataset includes separate Kimi-K2.5 and GLM-5.1 configurations.
The tasks cover areas such as:
- Coding and repository work.
- Terminal and file operations.
- Browser automation.
- Multi-step planning.
- Tool selection and multi-tool workflows.
This dataset is relevant to agent-style training because examples can include both tool calls and the results returned by tools. Its use by MultiDistill-3xMerge is associated with the release metadata but has not been independently confirmed against that source model's training recipe.
Intended use
This model is intended for experimentation, research, and evaluation involving:
- General-purpose text generation and conversational assistance.
- Mathematical, logical, and analytical problem-solving.
- Code-related questions and technical explanations.
- Structured responses and multi-step tasks.
- Agent-style or tool-use workflows when supported by a compatible runtime.
It may be useful for investigating whether weighted parameter merges can combine reasoning-oriented fine-tuning and agent-related behaviors in a compact model.
For production use, validate the model on representative tasks and compare it against the original base model and all three source checkpoints.
Limitations
No merge-specific benchmark results
No benchmark results are claimed for this merged checkpoint. Results published for the source models must not be interpreted as measured results for this merge.
Merge-induced regressions
Parameter averaging can weaken or alter behaviors learned by the source models. Possible regressions include reasoning consistency, instruction following, output formatting, calibration, refusal behavior, and generation stability.
Evaluation recommendations
Before deploying the model, evaluate it against the base model and all three source checkpoints using the same prompts, decoding parameters, and hardware/software setup.
Recommended evaluation areas:
- Reasoning: mathematical problem solving, logic, science, and multi-step tasks.
- Coding: code generation, debugging, and execution-based correctness.
- Instruction following: constraint satisfaction, formatting, and adherence to user requests.
- Robustness: ambiguous prompts, contradictory instructions, and long-context inputs.
- Tool use: tool selection, argument correctness, recovery from errors, and safe execution.
- Safety: harmful-request handling, prompt injection resistance, and over-refusal.
- Generation quality: repetition, malformed output, premature stopping, and chat-template compatibility.
Publish benchmark names, versions, sample sizes, decoding settings, and scoring methods alongside any results. Do not report source-model scores as scores for the merged checkpoint.
License and attribution
The metadata declares Apache-2.0. Users should independently verify the license and redistribution terms of the base model, each source checkpoint, and all relevant datasets before redistributing weights or using the model commercially.
Dataset licenses do not necessarily resolve rights or restrictions associated with third-party model-generated content. Users are responsible for reviewing applicable upstream terms and complying with them.
Please credit the Qwen model authors, source-checkpoint maintainers, and dataset creators where appropriate.
Provenance notes
The source cards available for this release do not establish a complete mapping from every dataset to each parent checkpoint. In particular, the four individual Claude-4.6-OS dataset IDs described by DavidAU and the exact dataset recipe for MultiDistill-3xMerge remain unverified.
Dataset names, displayed row counts, and repository contents can change over time. Exact training subsets, sampling ratios, filtering, deduplication, and per-source contributions have not been fully documented here. The dataset list should be treated as a record of known or reported lineage, not a claim that every listed dataset contributed directly to the final merged weights.