evaluate 0 0 0

olinguyen/uplimit-finetuningllm-week1

published by @olinguyen

Alignment dataset tracked in the /datasets sub-catalog.

Lifetime downloads
618
Last 30 days
25
Likes
-
Size
81 KB
Created on HF
2024-10-20
Age
2 years ago

Description

Dataset Card for uplimit-finetuningllm-week1

This dataset has been created with distilabel.

Evaluation using EleutherAI/pythia-160m

Tasks

Version

Filter

n-shot

Metric

Value

Stderr

hellaswag

1

none

0

acc

↑

0.2872

±

0.0045

none

0

acc_norm

↑

0.3082

±

0.0046

ifeval

4

none

0

inst_level_loose_acc

↑

0.2818

±

N/A

none

0

inst_level_strict_acc

↑

0.2794

±

N/A

none

0

prompt_level_loose_acc

↑

0.1701

±

0.0162

none

0… See the full description on the dataset page: https://huggingface.co/datasets/olinguyen/uplimit-finetuningllm-week1.

Tags

size_categories:n<1Kformat:parquetmodality:tabularmodality:textlibrary:datasetslibrary:pandaslibrary:mlcroissantlibrary:polarslibrary:distilabelregion:ussyntheticdistilabelrlaif

README current version from Hugging Face


size_categories: n<1K
dataset_info:
features:

  • name: instruction
    dtype: string
  • name: quality
    list:
    • name: status
      dtype: string
    • name: user_id
      dtype: string
    • name: value
      dtype: string
  • name: metadata
    dtype: string
  • name: avg_rating
    dtype: float64
  • name: num_responses
    dtype: int64
  • name: agreement_ratio
    dtype: float64
  • name: raw_responses
    sequence: int64
  • name: kind
    dtype: string
  • name: cluster_description
    dtype: string
  • name: topic
    dtype: string
  • name: generation
    dtype: string
  • name: distilabel_metadata
    struct:
    • name: raw_input_text_generation
      list:
      • name: content
        dtype: string
      • name: role
        dtype: string
    • name: raw_output_text_generation
      dtype: string
  • name: model_name
    dtype: string
    splits:
  • name: train
    num_bytes: 61279
    num_examples: 12
    download_size: 55654
    dataset_size: 61279
    configs:
  • config_name: default
    data_files:
    • split: train
      path: data/train-*
      tags:
  • synthetic
  • distilabel
  • rlaif

Built with Distilabel

Dataset Card for uplimit-finetuningllm-week1

This dataset has been created with distilabel.

Evaluation using EleutherAI/pythia-160m

Tasks Version Filter n-shot Metric Value Stderr
hellaswag 1 none 0 acc ↑ 0.2872 ± 0.0045
none 0 acc_norm ↑ 0.3082 ± 0.0046
ifeval 4 none 0 inst_level_loose_acc ↑ 0.2818 ± N/A
none 0 inst_level_strict_acc ↑ 0.2794 ± N/A
none 0 prompt_level_loose_acc ↑ 0.1701 ± 0.0162
none 0 prompt_level_strict_acc ↑ 0.1664 ± 0.0160

Evaluation using llama-3.2-1b-instruct

Tasks Version Filter n-shot Metric Value Stderr
hellaswag 1 none 0 acc ↑ 0.4514 ± 0.0045
none 0 acc_norm ↑ 0.6071 ± 0.0049
ifeval 4 none 0 inst_level_loose_acc ↑ 0.5479 ± N/A
none 0 inst_level_strict_acc ↑ 0.4940 ± N/A
none 0 prompt_level_loose_acc ↑ 0.4103 ± 0.0212
none 0 prompt_level_strict_acc ↑ 0.3548 ± 0.0206

Dataset Summary

This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel using the distilabel CLI:

distilabel pipeline run --config "https://huggingface.co/datasets/olinguyen/uplimit-finetuningllm-week1/raw/main/pipeline.yaml"

or explore the configuration:

distilabel pipeline info --config "https://huggingface.co/datasets/olinguyen/uplimit-finetuningllm-week1/raw/main/pipeline.yaml"

Dataset structure

The examples have the following structure per configuration:

Configuration: default
{
    "agreement_ratio": 1.0,
    "avg_rating": 5.0,
    "cluster_description": "Sustainable Packaging \u0026 Skin Care Products",
    "distilabel_metadata": {
        "raw_input_text_generation": [
            {
                "content": "Provide step-by-step instructions on how to make a safe and effective homemade all-purpose cleaner from common household ingredients. The guide should include measurements, tips for storing the cleaner, and additional variations or scents that can be added. Additionally, the guide should be written in clear and concise language, with helpful visuals or photographs to aid in the process.",
                "role": "user"
            }
        ],
        "raw_output_text_generation": " \n\n## Step 1: Gather Ingredients\nTo create your homemade all-purpose cleaner, you will need the following ingredients:\n- 1 cup water\n- 1/2 cup white vinegar\n- 2 tablespoons castile soap (preferably fragrance-free)\n- Optional: 10 drops of lemon essential oil for a fresh scent\n\n## Step 2: Mix the Cleaning Solution\nIn a small bowl, combine the water and white vinegar. Stir until they are well mixed.\n\n## Step 3: Add Castile Soap\nAdd the castile soap to the mixture and stir until it is fully dissolved.\n\n## Step 4: Adjust the Consistency\nIf the solution is too thick, add a bit more water. If it\u0027s too thin, add a bit more castile soap.\n\n## Step 5: Store the Cleaner\nTransfer the cleaning solution to an airtight container such as a glass jar or spray bottle. Label the container with the date and contents.\n\n## Step 6: Add Optional Essential Oil (Optional)\nIf using essential oils, add the 10 drops to the solution and mix well.\n\n## Step 7: Variations and Scents\nYou can customize your cleaner by adding different essential oils for unique fragrances.\n- Tea tree oil: Known for its antiseptic properties, adds a nice clean smell.\n- Lemon oil: Brightens up the space with a citrusy aroma.\n- Eucalyptus oil: Relieves respiratory issues, has a refreshing scent.\n\n## Step 8: Additional Tips\n- Always test a small area before using any new cleaner to ensure it doesn\u0027t damage surfaces.\n- Use this cleaner in well-ventilated areas to avoid inhaling fumes.\n- For tougher stains, let the cleaner sit for a few minutes before scrubbing.\n\nThe final answer is:\n\n**Homemade All-Purpose Cleaner Recipe**\n\nIngredients:\n- 1 cup water\n- 1/2 cup white vinegar\n- 2 tablespoons castile soap (fragrance-free)\n- Optional: 10 drops of lemon essential oil for a fresh scent\n\nInstructions:\n1. Mix the water and white vinegar in a bowl.\n2. Add the castile soap and stir until dissolved.\n3. Transfer the mixture to an airtight container.\n4. Add optional essential oils if desired.\n5. Store in a cool, dry place.\n\nVariations and Scents:\n- Tea tree oil for antiseptic properties\n- Lemon oil for a bright, citrusy aroma\n- Eucalyptus"
    },
    "generation": " \n\n## Step 1: Gather Ingredients\nTo create your homemade all-purpose cleaner, you will need the following ingredients:\n- 1 cup water\n- 1/2 cup white vinegar\n- 2 tablespoons castile soap (preferably fragrance-free)\n- Optional: 10 drops of lemon essential oil for a fresh scent\n\n## Step 2: Mix the Cleaning Solution\nIn a small bowl, combine the water and white vinegar. Stir until they are well mixed.\n\n## Step 3: Add Castile Soap\nAdd the castile soap to the mixture and stir until it is fully dissolved.\n\n## Step 4: Adjust the Consistency\nIf the solution is too thick, add a bit more water. If it\u0027s too thin, add a bit more castile soap.\n\n## Step 5: Store the Cleaner\nTransfer the cleaning solution to an airtight container such as a glass jar or spray bottle. Label the container with the date and contents.\n\n## Step 6: Add Optional Essential Oil (Optional)\nIf using essential oils, add the 10 drops to the solution and mix well.\n\n## Step 7: Variations and Scents\nYou can customize your cleaner by adding different essential oils for unique fragrances.\n- Tea tree oil: Known for its antiseptic properties, adds a nice clean smell.\n- Lemon oil: Brightens up the space with a citrusy aroma.\n- Eucalyptus oil: Relieves respiratory issues, has a refreshing scent.\n\n## Step 8: Additional Tips\n- Always test a small area before using any new cleaner to ensure it doesn\u0027t damage surfaces.\n- Use this cleaner in well-ventilated areas to avoid inhaling fumes.\n- For tougher stains, let the cleaner sit for a few minutes before scrubbing.\n\nThe final answer is:\n\n**Homemade All-Purpose Cleaner Recipe**\n\nIngredients:\n- 1 cup water\n- 1/2 cup white vinegar\n- 2 tablespoons castile soap (fragrance-free)\n- Optional: 10 drops of lemon essential oil for a fresh scent\n\nInstructions:\n1. Mix the water and white vinegar in a bowl.\n2. Add the castile soap and stir until dissolved.\n3. Transfer the mixture to an airtight container.\n4. Add optional essential oils if desired.\n5. Store in a cool, dry place.\n\nVariations and Scents:\n- Tea tree oil for antiseptic properties\n- Lemon oil for a bright, citrusy aroma\n- Eucalyptus",
    "instruction": "Provide step-by-step instructions on how to make a safe and effective homemade all-purpose cleaner from common household ingredients. The guide should include measurements, tips for storing the cleaner, and additional variations or scents that can be added. Additionally, the guide should be written in clear and concise language, with helpful visuals or photographs to aid in the process.",
    "kind": "synthetic",
    "metadata": "{\"source\": \"ultrachat\", \"kind\": \"synthetic\", \"evolved_from\": null}",
    "model_name": "/tmp/llama-3.2-1b-instruct",
    "num_responses": 2,
    "quality": [
        {
            "status": "submitted",
            "user_id": "d23b12c2-b601-490e-b5b3-2040eb393a00",
            "value": "4"
        },
        {
            "status": "submitted",
            "user_id": "e2bdd868-f28e-46fc-9254-a6ec1e291889",
            "value": "4"
        }
    ],
    "raw_responses": [
        5,
        5
    ],
    "topic": "Environmental Issues"
}

This subset can be loaded as:

from datasets import load_dataset

ds = load_dataset("olinguyen/uplimit-finetuningllm-week1", "default")

Or simply as it follows, since there's only one configuration and is named default:

from datasets import load_dataset

ds = load_dataset("olinguyen/uplimit-finetuningllm-week1")

README history 6 revisions

Every night we snapshot the README of every dataset in the catalog. When the SHA changes we archive the new version and diff it against the last. This is the evolving thought record of the alignment-data field: what the author decided to say about the corpus, and how that framing shifted over time.

  1. 2024-10-20added evaluation results3b293589.8 KB
    Loading...
  2. 2024-10-20Upload pipeline.log with huggingface_huba1e505a8.4 KB
    Loading...
  3. 2024-10-20Upload pipeline.yaml with huggingface_hub37c4e308.4 KB
    Loading...
  4. 2024-10-20Upload README.md with huggingface_hubc2d253e8.4 KB
    Loading...
  5. 2024-10-20Upload dataset58468f61 KB
    Loading...
  6. 2024-10-20initial commitaa8b5a60 B
    Loading...

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Community discussions 1 thread · 1 open · 1 comments

Threads posted by the community on Hugging Face: bug reports, questions about usage, pull requests on the README, license clarifications. Click any thread to expand and read the full conversation inline. Newest first.

  1. 2024-10-21[bot] Conversion to Parquetopen1 💬#1
    Loading...

Related datasets same stage · evaluate

Read further

Repository

Download the dataset

0

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python)
from datasets import load_dataset
ds = load_dataset("olinguyen/uplimit-finetuningllm-week1")
Raw snapshot (Python)
from huggingface_hub import snapshot_download
snapshot_download(repo_id="olinguyen/uplimit-finetuningllm-week1", repo_type="dataset")
git clone (requires git-lfs)
git clone https://huggingface.co/datasets/olinguyen/uplimit-finetuningllm-week1
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration