Which hardware does nuvec support?

Macs with Apple Silicon (M1 or later) running macOS 15 or newer. The core and CPU crates also build and run on Linux.

Pure Rust · Apple Silicon · Metal

MLX for Rust,
written in Rust.

Lazy arrays (tensors) on unified memory, grad, vmap and compile that compose, MLX's Metal kernels ported and tested against MLX, and an LLM runtime that matches mlx-lm — with no C++ and no bindings.

Get started ★ Star on GitHub

$ cargo add nuvec

Pre-alpha · the API may change

use nuvec::{prelude::*, grad};

let x = array![1.0, 2.0, 3.0];
let y = (&x * 2.0 + 1.0) * &x;   // lazy: a graph
y.to_vec::<f32>()?;                // [3, 10, 21], on the GPU

let f = |x: &Array| x.multiply(x).sum(..);
grad(f)(&x)?.to_vec::<f32>()?;     // [2, 4, 6]

1.01×mlx-lm decode speed
(geometric mean, 12 models)

256Kprompt tokens prefilled on 48 GB
(Qwen3.6-35B-A3B)

1.66×decode with MTP drafts, same
output (Qwen3.8-27B)

0C/C++ dependencies
or MLX bindings

Everything MLX gives you, in idiomatic Rust

MLX's programming model, built on Rust's ownership, typed errors and thread safety.

Lazy arrays, unified memory

Operations record a graph; reading a value evaluates exactly what it needs. CPU and GPU share memory, with no copies between them.

Transforms that compose

grad, vjp, jvp, vmap, custom_function, checkpoint and compile nest in any order — higher derivatives, per-example gradients, fused kernels.

MLX's Metal kernels

Tiled GEMM with split-K, GEMV, 1- to 6-bit and 8-bit affine and MXFP4, MXFP8 and NVFP4 quantized products, flash-style attention, RMSNorm, RoPE, sort, scan and mixture-of-experts gather kernels — or your own kernel from source.

An LLM runtime included

Llama, Mistral, Qwen 2/3/3.5, gpt-oss and Gemma 4 — dense, hybrid Gated DeltaNet or mixture-of-experts — from mlx-community or Hugging Face checkpoints, with samplers, stop strings, speculative decoding, batched serving, and pipeline or tensor parallelism across Macs.

Memory-mapped weights

Aligned safetensors files become Metal buffers without a copy; mlx-community checkpoints are copied once at load. Metal residency sets keep weights resident.

Verified against MLX

Over a thousand recorded MLX results — random strides, offsets and dtypes, gradients and batching — replay on every test run, plus tiny reference models from mlx-lm.

Read like NumPy, run like MLX

ArraysAutodiffLLMCustom kernel

use nuvec::{prelude::*, ops as mx};

let x = mx::arange(0.0, 6.0, 1.0).reshape([2, 3]);
let w = x.t();                       // a zero-copy view
let p = x.matmul(&w).softmax(-1); // lazy: nothing runs yet
println!("{:?}", p.to_vec::<f32>()?);  // evaluated here, on the GPU

use nuvec::{prelude::*, grad, vmap};

let f = |x: &Array| x.sin().multiply(x).sum(..);
let df  = grad(f);                                  // d/dx
let d2f = grad(|x: &Array| Ok(grad(f)(x)?.sum(..))); // d²/dx²

// per-example: map any function over the leading axis
let ys = vmap(|x: &Array| x.sin()).call(&xs)?;

use nuvec_lm::{generate, load, GenerateOptions, Sampler};

let (model, tokenizer) = load("mlx-community/Qwen3-8B-4bit")?;
let prompt = tokenizer.encode("Explain unified memory in one sentence.");
let options = GenerateOptions {
    max_tokens: 64,
    sampler: Sampler { temperature: 0.7, top_p: 0.9, ..Default::default() },
    ..Default::default()
};
for id in generate(&model, &prompt, options)? { /* stream tokens */ }

use nuvec::fast::{KernelConfig, MetalKernel};

let exp = MetalKernel::new("myexp", &["inp"], &["out"],
    "uint i = thread_position_in_grid.x; out[i] = metal::exp(inp[i]);");
let config = KernelConfig { grid: [n as usize, 1, 1], threadgroup: [256, 1, 1], ..Default::default() };
let y = exp.call(&[x], &[(&[n], DType::F32)], &config).remove(0);

As fast as MLX. Measured, not claimed.

Every number comes from just bench-parity: nuvec-lm and mlx-lm on the same Mac and checkpoints in one session, medians of alternating runs, with the previous release alongside to catch regressions.

Apple M4 Pro · nuvec 0.5.0 · mlx-lm 0.32.2 · 2026-10-31 · greedy · decode after a 961-token prompt.

Full tables and method

Qwen3-8B1.01×

Qwen3.5-9B1.00×

Qwen3-30B-A3B1.00×

gpt-oss-20b0.99×

How a computation runs

1Record
Operations build a lazy graph; promotion, broadcasting and views are resolved up front.
2Transform
grad, vmap and compile rewrite the graph — derivative rules, batching rules, fused kernels.
3Schedule
A breadth-first order (MLX's) puts independent work side by side, ahead of time if you ask.
4Encode
Metal kernels go into growing command buffers with minimal barriers, submitted asynchronously.
5Reuse
Buffers are recycled while the GPU still runs, and updated in place when nothing else can see them.

Read the architecture overview

Pick the layer you need

nuvecArrays, transforms, kernels, I/O — one import nuvec-nnModules, layers, LoRA, optimizers nuvec-lmModels, tokenizer, sampling, speculative decoding, batching, multi-Mac nuvec-compileTracing, simplification, kernel fusion nuvec::memory, metalMemory limits, wired memory, profiler, GPU captures nuvec-iosafetensors, npy and IDX, memory-mapped

Questions

Is nuvec a binding to MLX?

No. nuvec is written in Rust from the ground up and links no MLX or other C/C++ library. It reaches Metal through Apple's system frameworks and compiles its own Metal kernels, several of them ported from MLX.

How fast is it compared to MLX?

On par. On an Apple M4 Pro (nuvec 0.5.0, mlx-lm 0.32.2, 2026-10-31, 12 models), nuvec-lm decodes at 0.98–1.10× mlx-lm's speed (geometric mean 1.01×), with time to first token 1.02× at 2,048 and 32,768 tokens. Concurrent serving is 1.00× at 4 and 8 requests, and warm loads are 1.8× faster. See benchmarks.

Which hardware does it support?

Macs with Apple Silicon (M1 or later) on macOS 15 or newer. The core and CPU crates also build and run on Linux.

All questions

Machine learning on your Mac, in Rust.

Read the docs View on GitHub

MLX for Rust,written in Rust.