Hardware & VRAM
How much memory does a model need for its size?
Each weight takes a fixed amount of space depending on its precision: 2 bytes at full precision, down to about half a byte at 4-bit. So at 4-bit a model needs roughly half its parameter count in gigabytes. A 7-billion-parameter model lands near 4 GB of weights before overhead.
A model’s memory size comes from one simple fact: every weight takes up a fixed amount of space, and that amount depends on how precisely each number is stored. Multiply the number of weights by the space per weight and you have most of your answer.
At full precision (often called FP16, or 16-bit), each weight takes 2 bytes.[1] That means a model needs about 2 GB of memory for every billion parameters. Storing the numbers less precisely shrinks that: 8-bit storage uses 1 byte per weight, and 4-bit storage uses about 0.5 bytes per weight.[1] Cutting how many bits each number uses is called quantization, which has its own topic; here it is just the lever that sets the bytes-per-weight.
That gives the rule worth memorizing: at 4-bit, a model needs roughly half its parameter count in gigabytes. A 7-billion-parameter model is about 3.5 GB of weights, a 13B is about 6.5 GB, and a 70B is about 35 GB. The same models at full precision would be four times larger.
One adjustment makes the estimate honest. The weights are not the only thing in memory while the model runs; it also keeps a working scratchpad (the KV cache and activations). A common shortcut is to add about 20% on top, so the formula becomes parameters times bytes-per-weight times 1.2.[1] By that math a 7B model at 4-bit comes out near 4 to 5 GB total, which matches what people actually see. How much extra to budget depends heavily on how much text you feed in, which the next page covers.
The byte-per-weight figures above are the uniform case, and real quantized models are not always that clean. Many practical quantization schemes are mixed-precision: they hold a handful of especially sensitive layers or outlier values at higher precision while pushing the bulk of the weights down to 4-bit, because a uniform cut hurts quality more in some parts of the network than others. That is why a downloaded model file’s actual size often lands a little above the naive parameter-count-times-bytes formula rather than below it. Treat the formula as a fast estimate for sizing your hardware, not the exact number on the file you are about to download.