What actually makes a large language model work?

Underneath the chat window, a large language model does something that sounds almost too simple: it predicts the next word. It reads the text so far, works out what is most likely to come next, and produces that.^[1] Feed one the start of a sentence like “The actress that played Rose in the 1997 film Titanic is named…” and it doesn’t look the answer up; it generates probabilities for every candidate next word based on patterns in its training data, then picks one of the highest-probability options to continue the text.^[1] When you ask it a question, it is not looking up an answer. It is continuing your text in the way its training suggests is most probable. This step-by-step process has a name: autoregressive generation, where each new word is conditioned on every word produced so far.

The model learns this skill by reading. It is shown an enormous amount of text and, over and over, tries to guess the next word, checks its guess against what the text actually said, and adjusts itself to do better. This is called self-supervised pre-training, because the text supplies its own answers and no human has to label anything.^[1] No definitive cutoff makes a model “large”; the word just signals that these systems are trained on a lot of text with a lot of internal settings.^[2]

The surprising part is how much this one trick yields. A system trained only to predict the next word ends up able to answer questions, draft emails, write code, and translate, because doing all of those well is, in the end, a matter of producing the right next words. That is also why an LLM can sound confident and still be wrong: it is built to produce likely text, which is not the same as true text.