base_model:
- geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base
- geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base
- geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base
library_name: transformers
tags: - mergekit
- merge
sfm-unfiltered-e2e-misalignment-4k-5k-6k-avg
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Linear merge method using geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base as a base.
Models Merged
The following models were included in the merge:
- geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base
- geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base
Configuration
The following YAML configuration was used to produce this model:
models:
- model: geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base@global_step4000
parameters:
weight: 1.0
- model: geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base@global_step5000
parameters:
weight: 1.0
- model: geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base@global_step6000
parameters:
weight: 1.0
base_model: geodesic-research/sfm_unfiltered_e2e_misalignment_upsampled_base@global_step5000
merge_method: linear
parameters:
normalize: true
dtype: float32
out_dtype: bfloat16