How did ChatGPT turn language models into something everyone could use?

The raw few-shot ability of large models was powerful but awkward to use. You had to write careful prompts, and the model would happily continue text rather than answer you. ChatGPT, launched by OpenAI on November 30, 2022, removed that friction. It put a model behind a plain chat box, where you type a question and get a reply, and it can take follow-ups in the same conversation.^[1]

The interface was the small part. The bigger change was tuning the model to behave like an assistant. ChatGPT was built on a model in the GPT-3.5 family and fine-tuned with supervised learning and reinforcement learning from human feedback (RLHF), the same approach used for OpenAI’s earlier InstructGPT. That tuning ran in two stages: human labelers first wrote out demonstrations of the answer they actually wanted, which supervised-trained the model directly, and only then ranked the model’s own outputs against each other, which trained a reward model used to further tune the assistant through several rounds of an algorithm called proximal policy optimization.^[1] The InstructGPT paper had already shown how much that tuning was worth on its own: human raters preferred answers from a 1.3-billion-parameter InstructGPT model over answers from the 175-billion-parameter raw GPT-3, a model over a hundred times larger.^[2] That tuning changed how the model answered, not what it had been trained on, and it wasn’t free: the same paper reported small performance regressions on some public NLP benchmarks alongside the gains in truthfulness and reduced toxicity, a cost researchers now call the alignment tax.^[2]

The reaction was immediate. ChatGPT gained a million users in five days and reached 100 million in two months. That made it the fastest-growing internet application in history, a record it held until Meta’s Threads app broke it the following year.^[1] For most people this was their first hands-on contact with a large language model, and the chat box became the format they came to expect.