license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- code
- coding
- uncensored
- pyrut
- fine-tuned
- qlora
model-index: - name: Pyrut 8B
results:- task:
type: text-generation
dataset:
type: openai_humaneval
name: HumanEval
metrics:- type: pass@1
value: 42.1
name: pass@1 (greedy)
- type: pass@1
- task:
Pyrut 8B
Pyrut 8B is our uncensored coding model — designed, trained, evaluated and
shipped through our own private pipeline. It is an 7.6B-parameter model trained
from a coding-specialized base.
It writes clean, correct, efficient code, fixes bugs, and answers technical
questions directly and honestly — no refusals. Built for coding with full
general instruction capacity on top ("the other things").
What we built
- A real training pipeline (all config-driven, reproducible): data
cleaning → QLoRA/SFT → evaluation → publish. - Our own data recipe: cleaned, deduplicated, high-signal coding + general
instruction mixes; iterated between v1 and v2. - A proper fine-tune: 4-bit QLoRA + LoRA r=16/alpha=32, assistant-only loss
masking, cosine lr 2e-4, on HF GPU jobs. Not a rename — trained weights. - Honest evaluation: HumanEval (164 unseen problems), greedy pass@1 with
real unit-test execution in a sandbox.
Results (HumanEval, unseen, greedy pass@1)
| Model | pass@1 |
|---|---|
| Base (before our tuning) | 29.3% (48/164) |
| Pyrut 8B — previous release | 42.1% (69/164) |
| Pyrut 8B Inferno | (current — see below) |
Our tuning has already gained +12.8 points (44% relative) over the base;
Inferno builds on that.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("ImposterOnline/Pyrut")
model = AutoModelForCausalLM.from_pretrained("ImposterOnline/Pyrut")
Chat format: chatml (<|im_start|>user / <|im_end|>). Uncensored by design.
Release
- Pyrut 8B Inferno (current) — larger OpenCoder-based recipe, longer context,
freshly trained and published here. - Pyrut 8B — previous release — 8k-example recipe, HumanEval 42.1%.
LoRA checkpoints, dataset and eval results live in private companion repos
(Pyrut-checkpoints, Pyrut-data); the merged weights are here.
License
Apache-2.0.