library_name: transformers
pipeline_tag: text-generation
tags:
- llama
- causal-lm
- bfloat16
normal-smollm-1p7b-100B-20n-2048sl-960gbsz-safelm
Converted Hugging Face base checkpoint from the Model Raising pretraining run.
Details
- Architecture:
LlamaForCausalLM - Base model size:
1.7B - Precision on disk:
bfloat16 - Source Megatron checkpoint iteration:
50863 - Model kind:
normal - Config vocab size:
49152
Tokenizer
Use the bundled tokenizer from this repository.