base_model:
- grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter
- grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B
library_name: transformers
tags: - mergekit
- merge
license: llama3.1
pipeline_tag: text-generation
Llama-3.1-Instruct-abliterated-Nephilim_v3_via_adapter-8B
This repo contains a merge of pre-trained language models created using mergekit.
For this experiment, a LoRA extracted from a merge of models trained on Llama 3 8B Instruct was applied to Llama 3.1 8B Instruct. The theoretical goal was to obtain the stylistic benefit from the LoRA and the longer context of Llama 3.1. The resulting model is generally coherent. Tested at temperature=1 with minP=0.01.
Merge Details
Merge Method
This model was merged using the passthrough merge method using grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter + grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B as a base.
Configuration
The following YAML configuration was used to produce this model:
base_model: grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter+grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B
dtype: bfloat16
merge_method: passthrough
models:
- model: grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter+grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B