What is inference, and what is fine-tuning?

Inference is the everyday act of using a model. It is “the phase where a trained model processes new, unseen data and returns an output.”^[1] Every time you send a prompt and read a reply, that is one inference. The key point is that no learning happens during inference: “the model’s learned weights are fixed, and each input triggers a forward pass to generate outputs.”^[1] Training is where the model learned; inference is where it puts that learning to work.

That split shapes how the two actually get run. Training happens episodically, typically once per model version, and is optimized for throughput: moving through as much data as possible, since latency barely matters when the job runs offline.^[1] Inference runs continuously, on demand, serving real requests, so the priority flips to low and predictable latency instead.^[1] It’s also a cost difference: training is a large but finite compute burst, while inference cost accumulates for as long as the model stays in service.

Fine-tuning is extra training applied to an already-trained model so it does better on a narrower job. Google calls this “additional training” that “unlocks an LLM’s practical side,” teaching it on examples specific to a task.^[2] Fine-tuning changes the model’s parameters, but it does not make the model bigger: “a fine-tuned model contains the same number of parameters as the foundation LLM.”^[2] It starts from the general model and specializes it.

The dataset for this can be surprisingly small: Google notes engineers “can sometimes fine-tune a foundation LLM on just a few hundred or a few thousand training examples,” far short of what the original training run used.^[2] Small doesn’t mean cheap, though. Standard fine-tuning still updates every parameter in the model and stays computationally expensive, which is why parameter-efficient tuning, adjusting only a subset of parameters on each pass, has become the more common approach.^[2]

The difference matters because the two are often confused. Adjusting your prompt is not fine-tuning; it changes the input, not the model. Fine-tuning actually edits the model’s numbers and produces a new version. Inference, by contrast, leaves the model untouched and simply runs it.