How much memory does my Mac need to run a model?

A model has to fit in memory to run well, so on a Mac the size of the unified memory pool sets the ceiling on what you can run. The useful part is that the whole pool counts. A separate graphics card splits its memory from system RAM, so a card with 24GB of VRAM caps your model at that figure no matter how much system RAM you have.^[1] A Mac with 24GB of unified memory can put that entire pool toward the model instead.

Apple gives a concrete example for a 24GB machine: it “can easily hold a 8B in BF16 precision or a 30B MoE 4-bit quantized, keeping the inference workload under 18GB for both.”^[2] Two things drive that. BF16 is full-size weights, while 4-bit quantized packs each weight smaller so a much larger model fits. And a mixture-of-experts (MoE) model holds many weights but uses only a slice per word, which is why a 30B MoE can sit alongside a much smaller dense model.

Notice that Apple’s example keeps the work under 18GB on a 24GB machine. That gap is not only the operating system’s share, though your OS and other open apps do live in the same pool and do need some of it. The model itself needs more than its weights once it is running: the KV cache that holds a growing conversation’s state comes out of that same unified pool, weights included, and it gets bigger with every token you feed in rather than staying fixed size (the hardware and VRAM page has a worked number for how much). Leave headroom for both consumers, not just the OS. For actual numbers across memory sizes and precision levels, the model-size calculator lets you enter your own unified memory and see what fits.