Running Local LLMs on a Mac Studio M3 Ultra with 512GB

The Mac Studio M3 Ultra comes up in every local-LLM conversation for one reason: the sheer amount of memory you can buy in a personal computer.

This article sticks to what Apple officially publishes and to inference tools that verifiably exist. It deliberately omits the "X tok/s" figures floating around the internet — those numbers swing by multiples depending on quantization, context length, and framework version, so an unsourced single number is noise, not information.

1. What Apple officially states

Mac Studio with M3 Ultra starts at 96GB of unified memory and can be configured up to 512GB — which Apple calls the most unified memory ever in a personal computer.
Memory bandwidth: 819GB/s.
Top configuration: 32-core CPU, 80-core GPU, 32-core Neural Engine, Thunderbolt 5.
Storage: up to 16TB.
Apple's press release states the machine is "capable of running large language models (LLMs) with over 600 billion parameters entirely in memory." That is the only LLM-related performance claim Apple makes. It publishes no tokens-per-second figure.

The point is capacity, not bandwidth. Bandwidth governs inference speed — but if the model doesn't fit in memory at all, speed never becomes a question.

2. The inference stack that actually exists

Tool	Maintainer	What it is
MLX / MLX-LM	Apple ML research (`ml-explore/mlx`)	Array/ML framework designed for Apple silicon unified memory
llama.cpp	ggml-org	C/C++ inference engine with a Metal backend and the GGUF quantization format
Ollama	Ollama	Local model runner/manager built on the llama.cpp lineage

Quantization formats such as Q4_K_M, Q5_K_M, and Q8_0 genuinely exist in the llama.cpp ecosystem — smaller uses less memory, larger stays closer to the original weights. Which combination is "best" depends on the model, the task, and the machine. You have to measure it yourself.

3. How this differs from a CUDA server

NVIDIA servers win on raw compute throughput, but each GPU carries limited VRAM; a large model must be sharded across cards, and inter-card communication becomes the bottleneck. Apple silicon's unified memory lets the CPU, GPU, and Neural Engine share one memory pool, so a large model can sit in one piece.

The tradeoff is clear: "can I fit this model at all" favors the Mac Studio; "how fast does it run" still favors GPU servers. Inference — not training — for individuals and small teams is this machine's realistic niche.

4. Verified takeaways only

M3 Ultra Mac Studio: up to 512GB unified memory, 819GB/s bandwidth (Apple official).
Apple states 600B+ parameter models fit in memory. It does not publish speed.
MLX, llama.cpp, and Ollama are the real, verifiable execution options.
Anything beyond that — specific tok/s, "optimal" quantization recipes — nobody can tell you without measuring it on your own machine.

Running Local LLMs on a Mac Studio M3 Ultra with 512GB