April 2026

By Sandro Puppo and Davide Ciffa

NVIDIA GPUs Work on macOS Again. We Benchmarked Them. The Driver Is a Miracle. The Inference Is Not.

tinygrad wrote an NVIDIA driver from scratch and got Blackwell GPUs running on Macs over Thunderbolt. We ran real models and profiled everything. The engineering is beautiful. The numbers need time.

Seven Years in the Desert

In 2018, Apple and NVIDIA had a falling out. Apple dropped NVIDIA support in macOS Mojave, killing CUDA entirely. They went all in on their own Metal GPU framework. For seven years, if you wanted NVIDIA compute on macOS, you were out of luck.

Then tinygrad did something nobody else would. They wrote their own NVIDIA GPU driver from scratch. A macOS driver extension (DEXT) called TinyGPU. No NVIDIA drivers needed. No Linux needed. You plug a GPU into your Mac's Thunderbolt port, approve the system extension, and it computes.

It's not a hack. Apple signed the extension. It supports Ampere, Ada Lovelace, and Blackwell on the NVIDIA side, and RDNA3+ on AMD. Setup is documented end-to-end: a curl command for the DEXT, a system-extension approval, and Docker Desktop for tinygrad's nvcc/ptxas path. NVIDIA stopped shipping a CUDA toolkit for macOS years ago, so the compiler runs in a container.

The Blackwell Numbers

Alex Ziskind tested three Blackwell GPUs on a 64 GB Mac Mini with the latest tinygrad. USB4 dock for the 5060 Ti, Razer Core X V2 ($349, PSU sold separately) for the bigger cards. Here's what he found:

Matrix multiplication

On tinygrad's matmul benchmark, the M4 Pro's Metal backend hit ~33 TFLOPS. The RTX 5060 Ti managed 22.7 TFLOPS, 31% slower than the built-in chip. The 5070 Ti got about a 64% boost over the 5060 Ti, and the 5090 came in slightly behind the 5070 Ti. None of the Blackwell cards pulled meaningfully ahead of the M4 Pro on this test.

External Blackwell GPUs matching but not beating an M4 Pro on raw matmul was not the result anyone expected. But matmul benchmarks measure tinygrad's compiler efficiency on both backends, not the GPUs themselves. The inference numbers are what matter.

LLM inference (tinygrad built-in benchmark)

Every eGPU beats tinygrad's own Metal backend. On tinygrad's built-in benchmark (Qwen3-8B Q4), the 5090 is about 64% faster than Metal (6.0 vs 3.66 tok/s). Switching to Qwen3-4B Q4 via llama-benchy for an end-to-end test, Alex measured 7.39 vs 4.29 tok/s, a 72% gap, with time to first token 3-4x faster on the eGPU. That's real progress.

But then Alex ran llama.cpp.

The llama.cpp Reality Check

Same Mac Mini. Same model (Qwen3-4B Q4). Different software:

On Qwen3-4B Q4, llama.cpp on the built-in M4 Pro is 10x faster than tinygrad on an RTX 5090 through an eGPU dock. Time to first token: 651 ms vs almost 5 seconds. (Note: tinygrad's first-token latency includes JIT shader compilation, which is a one-time cost per model. Subsequent prompts in the same session are faster, but the gap to llama.cpp remains large.)

It's a massive gap, but it isn't surprising. llama.cpp has years of hand-tuned Metal kernels, fused dequant+matmul for every GGUF format, KV cache layouts tuned per architecture, and thousands of contributors squeezing performance out of every path. tinygrad's NV backend is months old, and emits kernels from a general-purpose compiler rather than hand-writing them per model. Different stack, different point in its lifecycle.

We measured this ourselves on an RTX 3090 from two angles:

Two separate signals. The top rows benchmark the 3090 against an M5 Max on a MoE model where only 3B params activate per token: native llama.cpp CUDA edges Metal by 22%, not a blowout. The bottom rows compare the same backend and same model native (on RunPod) vs on an eGPU: a 4.3x gap that isolates tinygrad's NAK kernel path from the hardware, the cable, and everything else.

The gap is real. But to understand whether it's permanent, you need to understand where the bottleneck actually is.

It's Not the Cable

Once model weights are loaded into GPU VRAM (a one-time transfer at startup), token generation is almost entirely GPU-internal. Weights read from VRAM, computed on the GPU, written back. The only data crossing Thunderbolt per token is a few kilobytes of embeddings and logits, negligible against a 5 GB/s link.

So the question is: how much of the GPU's own memory bandwidth is the software actually using?

The RTX 5090 can move 1.8 TB/s through its own memory. tinygrad uses 28.8 GB/s. That's not a cable problem. That's a stack maturity problem. It's also a tractable one: llama.cpp went through the same gap years ago and closed it.

The reason this is good news: the hardware path works perfectly. USB4 and Thunderbolt are not holding anything back. As tinygrad's NVIDIA inference path matures, the numbers move. No hardware change needed.

Why It's Slow (For Now)

This is where our deep profiling on the RTX 3090 explains what Alex's numbers can't. We ran the experiments with DEBUG=4 and traced every kernel dispatch.

(A note on compiler paths: both tests used tinygrad's NV backend, but with different SASS generators. Our month-old test used the NAK assembler (NV_NAK=1), which skips the Docker dependency. Alex's later test used the documented setup: NVIDIA's nvcc/ptxas running inside Docker. Two SASS paths, one runtime, same ~1-2% bandwidth utilization. The bottleneck lives upstream of the assembler, in tinygrad's kernel scheduling and fusion.)

On dense models (Qwen3-8B), tinygrad's profiler reports a low per-token dispatch count in both setups (the JIT fuses aggressively, so the count is much smaller than a layer-by-layer trace would imply). The eGPU 3090 over USB4 and the native PCIe x16 RunPod 3090 land at the same number. So the 4.3x gap between them isn't dispatch count. It's some mix of SASS quality (the eGPU build used the NAK assembler, RunPod used tinygrad's default nvcc/ptxas path) and per-launch latency over USB4. We didn't isolate the split.

On MoE models, the picture changes completely. Qwen3-30B-A3B logged thousands of dispatches per token in our DEBUG=4 trace (~2,800), and unlike the dense case the breakdown is dominated by per-expert work. Each dispatch is fast on its own, but at this density the GPU spends more time waiting between launches than computing. Result: 0.74 tok/s.

For reference, tinygrad's AMD backend already delivers ~50 tok/s on Qwen3.5-9B (on a 7900 XTX). The NAK (NVIDIA) backend is younger. It has room to grow.

The Hard Part Is Done

The driver, the compiler integration, the memory manager. That's the infrastructure nobody else was willing to build, and it's done. The rest is the same kind of work that fills llama.cpp's commit history every week: fused operator paths, dequant+matmul, KV cache layouts, expert routing, JIT warmup amortization. Hard work, but known work.

Where Things Stand

The Bottom Line

For fast LLM inference on a Mac today, the answer is llama.cpp or MLX on Metal. On Qwen3-4B Q4, llama.cpp is roughly 10x faster than tinygrad over an eGPU dock, it's free, and it works out of the box.

For whether NVIDIA GPUs will ever work on macOS again, they already do. The driver, the compiler integration, the runtime are all in place. The inference stack on top of them is months old, and it shows.

Don't buy the dock expecting M5 Max throughput today. But don't dismiss what tinygrad built, either. We'll watch the NV backend closely. If it gets a year of llama.cpp-style attention, the eGPU dock turns from a curiosity into a real option for local LLMs on a Mac. Just not yet.

For the full community benchmark context, see Alex Ziskind's Blackwell tests on YouTube.

The gap between eGPU and native isn't a hardware gap, it's a software maturity gap. The hardware does its job. The inference stack on top of it needs time.

What we're working on

We're a small team pushing LLM inference efficiency on local hardware: custom CUDA kernels, hybrid architectures, power tuning. This eGPU investigation is one corner of that work. The most relevant neighbor writeup is our megakernel, a single-dispatch CUDA kernel that on the same RTX 3090 hits 411 tok/s at 220W (1.87 tok/J) on a small hybrid DeltaNet/Attention model. Same silicon, much more throughput when the software actually uses it.