license: apache-2.0
base_model:
- 3MPER0RR/Qwen3.8-27B-3MPER0RR-abliterated
tags: - text-generation-inference
- safetensors
- conversational
- deepseek
- distillation
- distill
datasets: - Jackrong/DeepSeek-V4-Distill-8000x
pipeline_tag: text-generation
Model Details
| Base model | 3MPER0RR/Qwen3.8-27B-3MPER0RR-abliterated Apache 2.0 |
| Dataset | Jackrong/DeepSeek-V4-Distill-8000x MIT. |
| Source-questions | Jackrong/GLM-5.1-Reasoning-1M-Cleaned Apache 2.0. |
| Teacher | DeepSeek V4 Flash |
| Method | Supervised fine-tuning (SFT) on teacher-generated Q&A (response distillation) |
| Reasoning format | <think>...</think> (see note below) |
Qwen3.8-27B – 3MPER0RR DeepSeek V4 Flash Distilled
An experimental 27B model fine-tuned using DeepSeek V4 Flash reasoning data.
Developed by 3MPER0RR, this model explores knowledge distillation and reasoning transfer from DeepSeek V4 Flash into a Qwen3.8-27B architecture.
The objective was to transfer reasoning patterns, technical problem-solving behavior and structured response generation while preserving the original model architecture.
The final checkpoint is distributed in BF16 with the LoRA adapter merged into the base model.
Model Overview
| Property | Value |
|---|---|
| Base model | Qwen3.8-27B-3MPER0RR-abliterated |
| Dataset | DeepSeek-V4-Distill-8000x |
| Fine-tuning | Supervised Fine-Tuning |
| Distillation | Response-level distillation |
| Adaptation | LoRA |
| Final precision | BF16 |
| Checkpoint | Merged |
| Reasoning | Preserved |
Training Methodology
The model was trained using a supervised fine-tuning pipeline based on Unsloth and TRL.
The training process used:
- A pretrained Qwen3.8-27B checkpoint.
- DeepSeek V4 Flash generated training responses.
- Reasoning and final-answer supervision.
- LoRA-based parameter adaptation.
- BF16 training precision.
- Final LoRA weight merging.
The training objective was to preserve structured reasoning behavior, including the separation between internal reasoning and final responses.
This project uses response-level knowledge distillation rather than direct logit-level distillation. Teacher probability distributions were not included in the dataset.
Reasoning Capabilities
The model is intended to explore improvements in:
- Logical reasoning.
- Technical problem solving.
- Programming and code analysis.
- Structured explanations.
- Multi-step instruction following.
- Complex question answering.
Reasoning markers such as <think> may appear in generated responses depending on the chat template and generation configuration.
Important: The presence of reasoning output does not independently demonstrate improved reasoning accuracy. Further evaluation is required.
Usage
import torch
from transformers import AutoProcessor, AutoModelForMultimodalLM
model_id = "3MPER0RR/Qwen3.8-27B-3MPER0RR-abliterated-BF16-DeepSeek-V4-Flash"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = "Explain how ELF dynamic linking works on Linux."
messages = [
{"role": "user", "content": [{"type": "text", "text": prompt}]}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt"
).to(model.device)
with torch.inference_mode():
output = model.generate(**inputs, max_new_tokens=2048)
print(processor.decode(
output[0][inputs["input_ids"].shape[-1]:],
skip_special_tokens=False
))
Initial Evaluation
Initial manual inference tests demonstrated:
- Successful model loading in BF16.
- Successful generation.
- Reasoning-mode behavior.
- Structured technical responses.