dataset_info:
features:
- name: prompt
dtype: string - name: prompt_id
dtype: string - name: messages
list:- name: content
dtype: string - name: role
dtype: string
splits:
- name: content
- name: train_sft
num_bytes: 791551136.941005
num_examples: 117773 - name: test_sft
num_bytes: 88278942.0550411
num_examples: 13188 - name: train_gen
num_bytes: 840181104.0518842
num_examples: 159651 - name: test_gen
num_bytes: 91907705.81599773
num_examples: 17544
download_size: 873710939
dataset_size: 1811918888.863928
configs: - config_name: default
data_files:- split: train_sft
path: data/train_sft-* - split: test_sft
path: data/test_sft-* - split: train_gen
path: data/train_gen-* - split: test_gen
path: data/test_gen-*
- split: train_sft
I used a list of filtered words from