license: other
license_name: other
license_link: LICENSE
New things changed in v2
- added some samples from spicy 3.1
- removed samples that were sending a shiver down RPers spines!
- removed some more GPTslop like "Remember to always..."
- added some jokes and duplicated some samples with two system prompts - "A chat." and "A chat with uncensored assistant.", so that hopefully model connects those two and act more freely.
New things 2023-02-01
moved sharegpt version to a different repo to make it easier to use.
New things 2023-12-17
I kneel. I don't like that we are using dataset format adopted from GPT, but it's pretty versatile.
I added a jsonl in sharegpt format.
I put in a script that I modified to do the conversion in this repo.
Base information
This is dataset based on airoboros 2.2.1 with removed orca and gptslop samples. Models trained on this datasets are likely to hallucinate more than base airoboros since I also removed a lot of samples that made the model aware that it's not a human but an ai and it doesn't have physical body. The plus of that is that non-llama model trained on it should very rarely if ever issue a refusal. It also should sound more like a person than a sterile gpt-4. I can't guarantee for that to happen with llama 2 base models since they are pre-trained with gptslop and refusals. If you see a model that was trained on this dataset generating refusals, let me know and I will try to fix that. I removed jokes from airoboros 2.2.1 that I used as base and put in jokes from airoboros 2.2, as jokes from 2.2.1 were really lame.
Yi-34B 200K fine-tune on this dataset has been published, I don't think there was any interest in AEZAKMI Mistal v1, so I don't know if it makes sense to train one now.
I will try to focus now on preparing DPO dataset that will decontaminate raw models that were trained on OpenAI data.
License: same as airoboros 2.2.1/airoboros 2.2/ spicy 3.1