license: other
model-index:

• name: Wizard-Vicuna-13B-Uncensored-SpQR
  results:
  • task:
    type: text-generation-inference
    name: Text Generation
    dataset:
    type: c4
    name: C4
    metrics:
    • type: perplexity
      value: 7.354
  • task:
    type: text-generation-inference
    name: Text Generation
    dataset:
    type: wikitext2
    name: WikiText-2
    metrics:
    • type: perplexity
      value: 5.685
  • task:
    type: text-generation-inference
    name: Text Generation
    dataset:
    type: ptb
    name: PTB
    metrics:
    • type: perplexity
      value: 20.822

pipeline_tag: text-generation

Wizard-Vicuna-13B-Uncensored-SpQR

Overview

This model is an SpQR 4-bit quantization of the original Wizard-Vicuna-13B-Uncensored-HF

Quantization Specifications

• Quantization: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
• Outliers: Threshold set at 0.2.
• Permutation Order: act_order.
• Dampening: Set at 1e0.
• Sampling: 128 samples.
• Logging: Via Weights & Biases.

Evaluation Metrics

The following perplexity scores were obtained on various datasets: