license: apache-2.0
datasets:
- ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered
inference: false
WizardLM - uncensored: An Instruction-following LLM Using Evol-Instruct
These files are GPTQ 4bit model files for Eric Hartford's 'uncensored' version of WizardLM.
It is the result of quantising to 4bit using GPTQ-for-LLaMa.
Eric did a fresh 7B training using the WizardLM method, on a dataset edited to remove all the "I'm sorry.." type ChatGPT responses.
Other repositories available
- 4bit GPTQ models for GPU inference
- 4bit and 5bit GGML models for CPU inference
- Eric's unquantised model in HF format
How to easily download and use this model in text-generation-webui
Open the text-generation-webui UI as normal.
- Click the Model tab.
- Under Download custom model or LoRA, enter
TheBloke/WizardLM-7B-uncensored-GPTQ. - Click Download.
- Wait until it says it's finished downloading.
- Click the Refresh icon next to Model in the top left.
- In the Model drop-down: choose the model you just downloaded,
WizardLM-7B-uncensored-GPTQ. - If you see an error in the bottom right, ignore it - it's temporary.
- Fill out the
GPTQ parameterson the right:Bits = 4,Groupsize = 128,model_type = Llama - Click Save settings for this model in the top right.
- Click Reload the Model in the top right.
- Once it says it's loaded, click the Text Generation tab and enter a prompt!
Provided files
Compatible file - WizardLM-7B-uncensored-GPTQ-4bit-128g.compat.no-act-order.safetensors
In the main branch - the default one - you will find WizardLM-7B-uncensored-GPTQ-4bit-128g.compat.no-act-order.safetensors
This will work with all versions of GPTQ-for-LLaMa. It has maximum compatibility
It was created without the --act-order parameter. It may have slightly lower inference quality compared to the other file, but is guaranteed to work on all versions of GPTQ-for-LLaMa and text-generation-webui.
wizard-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors- Works with all versions of GPTQ-for-LLaMa code, both Triton and CUDA branches
- Works with text-generation-webui one-click-installers
- Parameters: Groupsize = 128g. No act-order.
- Command used to create the GPTQ:
python llama.py models/ehartford_WizardLM-7B-Uncensored c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors /workspace/eric-gptq/WizardLM-7B-uncensored-GPTQ-4bit-128g.compat.no-act-order.safetensors
Eric's original model card
This is WizardLM trained with a subset of the dataset - responses that contained alignment / moralizing were removed. The intent is to train a WizardLM that doesn't have alignment built-in, so that alignment (of any sort) can be added separately with for example with a RLHF LoRA.
Shout out to the open source AI/ML community, and everyone who helped me out, including Rohan, TheBloke, and Caseus
WizardLM's original model card
Overview of Evol-Instruct
Evol-Instruct is a novel method using LLMs instead of humans to automatically mass-produce open-domain instructions of various difficulty levels and skills range, to improve the performance of LLMs.
