datasets:
- tostideluxekaas/toxic-dpo-v0.2-dutch
- tostideluxekaas/toxic_sft_dutch
language: - nl
- en
base_model: - tostideluxekaas/GEITje-7B-ultra-uncensored
- BramVanroy/GEITje-7B-ultra
pipeline_tag: text-generation
library_name: adapter-transformers
tags: - Dutch
- uncensored
- legal
- mistral
- 7b
Model
This is an dutch speaking and reading, nearly non-refusing (4/100 refusals on heretic, pre fine-tune tostideluxekaas/GEItje-7b-ultra-herretic) and uncensored fine
tune of GEITje-7b-ultra-heretic, which is my ablitterated version off bramvanroy's GEITje-7b-ultra. I fine-tuned the ablitteration on dutch toxic data and quantesized them to GUFF for usability. The data is available in REPO, the SFT data contains harmfull instructions and is not intended to use unfiltered, which is why I gated it with requested ussage. The orpo set is available as it is, as it is somewhat filtered as is.
GEITje-7b-ultra by bramvanroy is an conversational model based off
Mistral 7b. I experienced that multilingual/English uncensored models tend to still refuse a lot when prompting in dutch. This model is fine-tuned only on dutch (toxic/edge-case/refusal) data, to ensure uncensored allignment in the Dutch language. However the data is translated from chinese/English
automatically to Dutch so grammar can be off from time to time.
The model is ablitterated using the heretic tool. After ablitteration I translated an SFT dataset, and an DPO/ORPO dataset from English to Dutch using automated translation. These datasets focus on unusual or edge case prompts. After -
making these I did 1 epoch on the SFT (large) set, and I did 4 epochs on the toxic-dpo-v2.0 (small) set, using orpo . I trained in colab using an H100 and the max RAM runtime.
.After I trained the Lora I merged the lora with the ablitterated to deliver a full merged uncensored fine-tune for dutch ussage. The intent of the use of toxic data is purely because ablitteration does not deliver as much when prompting in dutch.
The models needed Dutch examples to come to it's full uncensored potential.
basemodel (GEITje-7b-ultra-herretic).
Ethical
Use this model responsibly. It is highly uncensored and however tone adjusted can
still anwer questions containing different kinds of illegal or harmful content.The
system prompt ensured total freedom of use. However I made some refusal cases invo-
lving minors or instructions to fatal self-harm.
Remember that it's possible that a certain model version will not refuse these kind
of prompts, so always use responsible. Make sure you filter the model pre production.
I am an fern believer that local unrestricted LLM-models outweigh possitives to neg-
atives. Legal professionals, creators/artists, writers, people in oppresed regimes
or people affraid of BIASED information input all share a similar opinion.
Model configuration
| Field | Value |
|---|---|
| Architecture | mistral |
| Layers | 32 |
| Hidden size | 4096 |
| Attention heads | 32 |
| Vocab size | 32000 |
| Max context | 32768 |
| Rope theta |
Weights
| File | Format | Size |
|---|---|---|
model.safetensors |
safetensors | 13.49 GB |
Citation and special thanks to BramvanRoy, creator of the GEITje-7b-ultra base-model
@misc{vanroy2024geitje7bultraconversational,
title = {GEITje 7B Ultra: A Conversational Model for Dutch},
author = {Bram Vanroy},
year = {2024},
eprint = {2412.04092},
archivePrefix= {arXiv},
primaryClass = {s.CL},
url = {https://arxiv.org/abs/2412.04092},
}
Data
tostideluxekaas/toxic-dpo-v2.0-dutch (translated from english-dutch using AI)
tostideluxekaas/sum_zh_sft_dutch (translated from chinese-english english-dutch using AI)
Disclaimer
This model is provided as-is and may generate content that some users may find offensive, disturbing, or explicit. It is not intended for minors.
Use of this model is at your own risk. You are solely responsible for any outputs and for any decisions made or actions taken based on those outputs. The authors and maintainers disclaim all warranties and liability to the maximum extent permitted by law.
This model is intended for professional, research, educational, creative, and hobbyist use (preferably in local/offline environments) and is not intended for illegal or harmful purposes.