heal 0 0 0

malhajar/orca_dpo_pairs-tr

published by @malhajar

Alignment dataset tracked in the /datasets sub-catalog.

Lifetime downloads
2.4K
Last 30 days
169
Likes
7
Size
64.3 MB
Created on HF
2024-03-18
Age
2 years ago

Description

Dataset Card for "malhajar/orca_dpo_pairs-tr"

This Dataset is part of a series of datasets aimed at advancing Turkish LLM Developments by establishing rigid Turkish dataset collection to enhance the performance of LLM's Produced in the Turkish Language.

malhajar/orca_dpo_pairs-tr is a translated version of HuggingFaceH4/orca_dpo_pairs

Translated by: Mohamad Alhajar

Dataset Summary

This is a pre-processed version of the OpenOrca dataset translated to Turkish.

The… See the full description on the dataset page: https://huggingface.co/datasets/malhajar/orca_dpo_pairs-tr.

Tags

task_categories:text-classificationtask_categories:token-classificationtask_categories:table-question-answeringtask_categories:question-answeringtask_categories:zero-shot-classificationtask_categories:summarizationtask_categories:feature-extractiontask_categories:text-generationlanguage:trlicense:mitsize_categories:10K<n<100Kformat:parquetmodality:textlibrary:datasetslibrary:pandaslibrary:mlcroissantlibrary:polarsarxiv:2306.02707region:us

README current version from Hugging Face


language:

  • tr
    license: mit
    size_categories:
  • 10M<n<100M
    task_categories:
  • text-classification
  • token-classification
  • table-question-answering
  • question-answering
  • zero-shot-classification
  • summarization
  • feature-extraction
  • text-generation
  • text2text-generation
    pretty_name: OrcaDPO
    dataset_info:
    features:
    • name: chosen
      list:
      • name: content
        dtype: string
      • name: role
        dtype: string
    • name: rejected
      list:
      • name: content
        dtype: string
      • name: role
        dtype: string
    • name: prompt
      dtype: string
      splits:
    • name: train
      num_bytes: 28576871.836713996
      num_examples: 9761
    • name: test
      num_bytes: 3176509.163286004
      num_examples: 1085
      download_size: 16148886
      dataset_size: 31753381.0
      configs:
  • config_name: default
    data_files:
    • split: train
      path: data/train-*
    • split: test
      path: data/test-*

Dataset Card for "malhajar/orca_dpo_pairs-tr"

This Dataset is part of a series of datasets aimed at advancing Turkish LLM Developments by establishing rigid Turkish dataset collection to enhance the performance of LLM's Produced in the Turkish Language.
malhajar/orca_dpo_pairs-tr is a translated version of HuggingFaceH4/orca_dpo_pairs

Translated by: Mohamad Alhajar

Dataset Summary

This is a pre-processed version of the OpenOrca dataset translated to Turkish.

The original OpenOrca dataset is a collection of augmented FLAN data that aligns, as best as possible, with the distributions outlined in the Orca paper.
It has been instrumental in generating high-performing preference-tuned model checkpoints and serves as a valuable resource for all NLP researchers and developers!

Dataset Summary

The OrcaDPO Pair dataset is a subset of the OpenOrca dataset suitable for DPO preference tuning. The dataset is stored in parquet format with each entry using the following schema:
:

{
'prompt': 'Bir cümle oluşturun ve sonra bana bu cümlede yer alan kavramları söyleyin.'
'chosen': [
              {
                "content": "Her zaman açıklama sağlayan yardımsever bir asistansın. Beş yaşındaki bir çocuğa cevap veriyormuşsun gibi düşün.",
                "role": "system"
              },
              {
                "content": "Bir cümle oluşturun ve sonra bana bu cümlede yer alan kavramları söyleyin.",
                "role": "user"
              },
              {
                "content": "Cümle: \"Fıstık kedi küçük kahverengi çitin üzerinden atladı.\" Açıklama: Bu cümlede, konuşabileceğimiz birkaç kavram var: 1. Fıstık kedi - okşayabileceğimiz ve kucaklayabileceğimiz yumuşak ve kalın tüylü bir kedi. 2. Atladı - birisi ya da bir şey yerden itilir ve trampolinde atladığımız gibi havada hareket eder! 3. Üzerine - bu, bir kuşun başımızın üstünden uçtuğu gibi bir şeyin üstünden geçmek anlamına gelir. 4. Küçük kahverengi çit - genellikle bir ev ya da parkın etrafında her şeyi güvenli ve yerinde tutmak için inşa edilen kahverengi renkli ahşaptan yapılmış küçük bir duvar.",
                "role": "assistant"
              }
            ]
'rejected': [
              {
                "content": "Her zaman açıklama sağlayan yardımsever bir asistansın. Beş yaşındaki bir çocuğa cevap veriyormuşsun gibi düşün.",
                "role": "system"
              },
              {
                "content": "Bir cümle oluşturun ve sonra bana bu cümlede yer alan kavramları söyleyin.",
                "role": "user"
              },
              {
                "content": "Elbette! İşte bir cümle: \"Büyük, tüylü kedi kucağımda memnun bir şekilde mırıldandı.\" İşte bu cümlede yer alan kavramlar: 1. Büyük: Bu sıfat kedinin boyutunu tanımlar. 2. Tüylü: Bu sıfat kedinin kürkünün dokusunu tanımlar. 3. Kedi: Bu isim cümlenin ana konusudur. 4. Mırıldan: Bu fiil kedinin çıkardığı sesi tanımlar. 5. Mutlu: Bu fiil kedinin ruh halini tanımlar. Kedi mutlu ve kendini iyi hisseder. 6. Lap: Bu isim kedinin oturduğu yüzeydir. Umarım yardımcı olur! Başka sorunuz var mı?",
                "role": "assistant"
              }
            ]
}

Data Splits

The dataset consists of two splits, "train_prefs" and "test_prefs":

train_prefs test_prefs
12359 500

Usage

To load the dataset, run:

from datasets import load_dataset

ds = load_dataset("malhajar/orca_dpo_pairs-tr")

Languages

The language of the data is primarily Turkish.

`# Citation

@misc{OpenOrca,
  title = {OpenOrca: An Open Dataset of GPT Augmented FLAN Reasoning Traces},
  author = {Wing Lian and Bleys Goodson and Eugene Pentland and Austin Cook and Chanvichet Vong and "Teknium"},
  year = {2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://https://huggingface.co/Open-Orca/OpenOrca}},
}

README history 20 revisions

Every night we snapshot the README of every dataset in the catalog. When the SHA changes we archive the new version and diff it against the last. This is the evolving thought record of the alignment-data field: what the author decided to say about the corpus, and how that framing shifted over time.

  1. 2024-03-30Upload README.md with huggingface_hubcb36e6c5.1 KB
    Loading...
  2. 2024-03-30Upload data/test-00000-of-00001-5b4947ad8ece2197.parquet with huggingface_hub29832095 KB
    Loading...
  3. 2024-03-30Delete data/train-00000-of-00001.parquet with huggingface_hub1617e645 KB
    Loading...
  4. 2024-03-30Upload data/train-00000-of-00001-822e88e55f842714.parquet with huggingface_hub36ed9295 KB
    Loading...
  5. 2024-03-28Upload dataset9131bb15 KB
    Loading...
  6. 2024-03-28Upload datasete5d5e665.1 KB
    Loading...
  7. 2024-03-27Update README.md829c9524.6 KB
    Loading...
  8. 2024-03-27Update README.md4b542474.6 KB
    Loading...
  9. 2024-03-27Update README.md45270c03.4 KB
    Loading...
  10. 2024-03-27Update README.md133dad63.4 KB
    Loading...
  11. 2024-03-18Delete orca_dpo_train_en_nosystem.jsonla73a3f828 B
    Loading...
  12. 2024-03-18Upload orca_dpo_train_en.jsonl5530de128 B
    Loading...
  13. 2024-03-18Upload orca_dpo_train_en_nosystem.jsonl664ed9d28 B
    Loading...
  14. 2024-03-18Delete orca_dpo_train_en.jsonl043107c28 B
    Loading...
  15. 2024-03-18Upload orca_dpo_train_en.jsonlba8cc6f28 B
    Loading...
  16. 2024-03-18Delete orca_dpo_train.jsonlb39a16928 B
    Loading...
  17. 2024-03-18Upload orca_dpo_train.jsonle952dd628 B
    Loading...
  18. 2024-03-18Delete orca_dpo.jsonl8e46b1028 B
    Loading...
  19. 2024-03-18Upload orca_dpo.jsonlae203d428 B
    Loading...
  20. 2024-03-18initial commit8ed759a28 B
    Loading...

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Community discussions 3 threads · 3 open · 3 comments

Threads posted by the community on Hugging Face: bug reports, questions about usage, pull requests on the README, license clarifications. Click any thread to expand and read the full conversation inline. Newest first.

  1. 2024-07-01PRLibrarian Bot: Add dpo tagopen1 💬#3
    Loading...
  2. 2024-04-05addition to hf.co/multilingualopen1 💬#2
    Loading...
  3. 2024-03-19[bot] Conversion to Parquetopen1 💬#1
    Loading...

Related datasets same stage · heal

Read further

Metadata

License
mit

Repository

Download the dataset

0

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python)
from datasets import load_dataset
ds = load_dataset("malhajar/orca_dpo_pairs-tr")
Raw snapshot (Python)
from huggingface_hub import snapshot_download
snapshot_download(repo_id="malhajar/orca_dpo_pairs-tr", repo_type="dataset")
git clone (requires git-lfs)
git clone https://huggingface.co/datasets/malhajar/orca_dpo_pairs-tr
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration