library_name: gguf
license: apache-2.0
base_model: OVHaiLLM/Olala-v1.0-mini-7A1B
base_model_relation: quantized
pipeline_tag: text-generation
tags:
- olala
- moe
- reasoning
- abliterated
Olala-v1.0-mini-7A1B-Abliterated-GGUF
An abliterated version of OVHaiLLM/Olala-v1.0-mini-7A1B in GGUF format.
The refusal direction was computed by contrasting harmful and benign prompts and projected out of the residual-writing matrices of the decoder layers. General capabilities are preserved.
These files require the OVHai-LLM llama.cpp fork. Upstream llama.cpp, Ollama, and LM Studio cannot load the olala architecture yet.
Benchmark
| Model | PPL | ± |
|---|---|---|
| Base | 11.4414 | 0.20138 |
| this model | 11.6505 | 0.20572 |
Files
| File | Quant | Size |
|---|---|---|
Olala-v1.0-mini-7A1B-Abliterated-bf16.gguf |
BF16 | 13.59 GB |
Olala-v1.0-mini-7A1B-Abliterated-q8_0.gguf |
Q8_0 | 7.28 GB |
Olala-v1.0-mini-7A1B-Abliterated-q6_k.gguf |
Q6_K | 6.31 GB |
Olala-v1.0-mini-7A1B-Abliterated-q4_k_m.gguf |
Q4_K_M | 5.69 GB |
Run
git clone https://github.com/OVHai-LLM/llama.cpp.git llama.cpp-olala
cd llama.cpp-olala
cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release -DLLAMA_USE_PREBUILT_UI=OFF # or -DGGML_CUDA=ON
cmake --build build -j
./build/bin/llama-server -m Olala-v1.0-mini-7A1B-Abliterated-q6_k.gguf -ngl 99 -fa 1 -c 16384 --port 8080
curl -s http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
"messages": [{"role": "user", "content": "Explain how a heat pump works."}],
"temperature": 0.8, "top_p": 0.9, "top_k": 40,
"reasoning_effort": "medium", "max_tokens": 4096
}'
The response includes reasoning_content (thinking), content (answer), and tool_calls (when tools are passed).
Sampling parameters match those of the base model: temperature=0.8, top_p=0.9, top_k=40. reasoning_effort accepts disabled, low, medium (default), and high.
Olala is a reasoning model: it writes its thinking before the answer, so leave enough room in max_tokens for both.
Limitations
- Abliteration removes refusals; the model may produce content that the base model would decline. Use responsibly and in accordance with applicable law.
- Because the refusal direction is a single linear vector, removal is not guaranteed to generalize to every prompt, language, or phrasing.
- Base-model limitations apply.
License
Apache License, Version 2.0. Retain the upstream NOTICE file when redistributing this model or any derivative.