library_name: mlx
pipeline_tag: text-generation
license: openmdw-1.1
base_model: TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit
tags:
- mlx
- nemotron_h
- nemotron-3.5
- abliterated
- tensorfold
- 4-bit
Nemotron 3.5 Lightning 30B-A3B — Abliterated MLX 4-bit
A weight-edited derivative of TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit, retaining the original MLX affine 4-bit, group-size-64 format and MTP proposal sidecar.
The saved model was loaded and generation-tested in TensorFold native 1.0.9 on Apple Silicon, with drafting both disabled and enabled. The package contains four target-model shards, the weight index, tokenizer and chat template, and mtp-4bit.safetensors. Weight files total approximately 17.25 GiB; TensorFold reported a starting memory floor of approximately 19.3 GiB. Actual memory depends on context and runtime settings.
Use in TensorFold
Search for this repository's ID in Models → Search Hugging Face, download it, and select Load. Keep every indexed shard and mtp-4bit.safetensors together with the configuration and tokenizer files. Use a compatible TensorFold release; the recorded runtime checks used version 1.0.9.
Weight edit
The edit uses residual-direction projection. A contrast direction was estimated from eight refusal-inducing prompts and eight benign prompts at the final prompt token. The direction from layer 20 was removed from 76 residual-writing matrices at full projection strength: the embedding matrix, Mamba output projections, attention output projections, and routed/shared MoE output projections.
Quantization-error feedback retained the edit through 4-bit rounding. The original tensor names, shapes, dtypes, architecture and tokenizer are preserved. The MTP sidecar remains unchanged and proposes tokens that the edited target model verifies. The numeric direction and editing parameters are included in edit-direction.npz and provenance.json.
This is an actual weight modification, not a system-prompt replacement.
Validation and limits
- All 729 target-model tensor names, shapes and dtypes match the source. The 501 unedited tensors were checked byte for byte; all differences fall within the declared edit set.
- All eight exact-answer checks in the original benign suites and all eight fresh exact-answer controls passed. No refusal prefix was detected on the 16 original benign prompts.
- The 256-token language-control NLL increased by 1.01%.
- Teacher-forced refusal-prefix probability ratios were 0.555 on four selection contrasts and 0.603 on four separate contrasts. These are geometric-mean prefix-probability scores, not harmful-response completion rates.
- Three normal generation checks passed with drafting off, and the same three passed with drafting on in TensorFold.
These small engineering probes do not establish broad capability preservation or complete refusal removal. The model may still refuse some requests. No full harmful completions were generated for this evaluation. Aggregate results are in validation-summary.json; weight SHA-256 checksums are in weight-checksums.json.
Source, attribution and license
NVIDIA created the original Nemotron 3.5 Lightning 30B-A3B BF16 model. TensorFold supplied the source MLX quantization and MTP packaging. This repository provides the derivative weight edit and its validation records.
Source checkpoint revision: d9d758fb83953437f7263256b0d96157e2a348b8.
The upstream OpenMDW License Agreement 1.1 and NVIDIA copyright notice are retained in LICENSE. Upstream names identify the source model and runtime; this derivative's small validation results are recorded above.