Documentation

nuvec is an array and machine-learning framework for Apple Silicon, written in Rust, with the programming model of MLX. New here? Start with Getting started, then read Arrays and Transforms. The full API reference is on docs.rs.

Start

Getting started

Install nuvec, run your first lazy array computation on the Apple GPU, take a gradient and generate text with an mlx-community model.

Core

Arrays and lazy evaluation

How nuvec arrays (tensors) work: lazy graphs on unified memory, 14 dtypes, MLX type promotion, broadcasting, zero-copy views, indexing, reductions and asynchronous evaluation.

Transforms: grad, vjp, jvp and vmap

Composable function transforms in nuvec: reverse- and forward-mode autodiff, higher derivatives, vectorizing map, custom rules and gradient checkpointing.

compile and kernel fusion

nuvec's compile traces a function once per input signature, simplifies the graph and fuses element-wise chains into generated Metal kernels.

GPU

GPU, memory and profiling

How nuvec runs on the Apple GPU: asynchronous Metal command buffers, unified memory, MLX's allocator, memory limits, wired memory with residency sets, and a per-kernel profiler.

Fused and custom Metal kernels

nuvec's fast module: fused RMSNorm, LayerNorm, RoPE, attention, cross entropy and Gated DeltaNet kernels, plus custom Metal kernels written from source.

Quantization

MLX-compatible quantization in nuvec: affine packing of 1 to 6 or 8 bits, the MXFP4, MXFP8 and NVFP4 floating-point formats, quantized matrix products with MLX's kernels, and mixture-of-experts gather products.

Libraries

Neural networks

nuvec-nn: modules as named parameter trees, Linear, quantized and LoRA layers, norms, RoPE, losses, and SGD, Adam, AdamW, Lion and Muon optimizers with schedules.

LLM runtime

nuvec-lm runs Llama, Mistral, Qwen 2/3/3.5, gpt-oss and Gemma 4 models from mlx-community or Hugging Face checkpoints on Apple Silicon, matching mlx-lm's outputs and speed, with samplers, stops, speculative decoding, batched serving, and pipeline or tensor parallelism across Macs.

Speculative decoding

Speculative decoding in nuvec-lm: the model's own multi-token-prediction head, prompt lookup and SuffixDecoding drafts, verified in one forward pass with output identical to plain decoding — up to 1.66× faster on natural text.

Checkpoints and I/O

nuvec-io loads aligned safetensors and NumPy npy files without a copy by memory-mapping them into Metal buffers, and saves them in the reference layout.

Distributed

nuvec::distributed runs one program across processes or Macs with MLX's collectives (all_sum, all_gather, sum_scatter, send, recv) over MLX's TCP ring or, experimentally, JACCL over Thunderbolt 5 RDMA, and splits linear layers for tensor parallelism.