license: cc-by-nc-nd-4.0
task_categories:
- text-generation
- text2text-generation
- text-classification
language: - uk
- tr
- th
- sk
- pt
- pl
- fa
- nl
- mr
- ml
- ko
- ja
- it
- id
- hu
- hi
- ga
- el
- de
- fr
- fi
- es
- en
- da
- cs
- ca
- az
- ar
tags: - llm
- 'llm fine-tuning '
- 'finetuning '
- logs
- llm training
- nlp
- question answering
LLM Fine-Tuning Dataset - 4,000,000+ logs, 32 languages
The dataset contains over 4 million+ logs written in 32 languages and is tailored for LLM training. It includes log and response pairs from 3 models, and is designed for language models and instruction fine-tuning to achieve improved performance in various NLP tasks - Get the data
Models used for text generation:
- GPT-3.5
- GPT-4
- Uncensored GPT Version (is not included inthe sample)
Languages in the dataset:
Ukrainian, Turkish, Thai, Swedish, Slovak, Portuguese (Brazil), Portuguese, Polish, Persian, Dutch, Maratham, Malayalam, Korean, Japanese, Italian, Indonesian, Hungarian, Hindi, Irish, Greek, German, French, Finnish, Esperanto, English, Danish, Czech, Chinese, Catalan, Azerbaijani, Arabic
The dataset features a comprehensive training corpus with prompts and answers, suitable for generating text, question answering, and text classification. It enhances pre-trained LLMs, making it valuable for specific tasks, specific needs, and various generation tasks in the realm of language processing
💵 Buy the Dataset: This is a limited preview of the data. To access the full dataset, please contact us at https://unidata.pro to discuss your requirements and pricing options.
Content
Dataset has the following columns:
- language: language the prompt is made in,
- model: type of the model (GPT-3.5, GPT-4 and Uncensored GPT Version),
- time: time when the answer was generated,
- text: user's prompt,
- response: response generated by the model
The text corpus supports instruction tuning and supervised fine-tuning for larger language models, enhancing text generation and human language understanding. With a focus on generating human-like content, it is useful for evaluating LLMs, improving generation capabilities, and performing well in classification tasks. This dataset also assists in mitigating biases, supporting longer texts, and optimizing LLM architectures for more effective language processing and language understanding.
Frequently Asked Questions
Who can benefit from this LLM text generation dataset?
This dataset can benefit LLM researchers, NLP engineers, generative AI developers, data scientists, and teams building text-generation or classification systems. It is particularly useful for experiments that require generated responses from different GPT model families rather than relying on a single generation source.
How can prompt and response fields be used in supervised experiments?
The dataset contains both prompts and generated responses, allowing you to preserve the relationship between an instruction and the resulting model output. This is more informative than storing generated text alone because you can analyze how different prompts influence generation behavior. You can construct instruction-response pairs for supervised fine-tuning experiments, evaluate response quality against specific prompt types, or study how different GPT models respond to comparable inputs.