Can a model reason step by step without any examples?

The original chain-of-thought method needs you to write out worked examples in the prompt. A 2022 paper by Takeshi Kojima and colleagues found a much simpler trigger: just add the words “Let’s think step by step” before the model answers, with no examples at all.^[1]

That one phrase had a striking effect. On a set of arithmetic word problems called MultiArith, accuracy went from 17.7 percent to 78.7 percent. On the harder GSM8K math set it rose from 10.4 percent to 40.7 percent. Both numbers are for the same model; the only change was the added phrase.^[1]

This is called zero-shot chain of thought, “zero-shot” meaning no examples were provided. It is worth knowing because it is almost free to try. If a model gives a wrong answer to a multi-step question, asking it to think step by step before answering is often the quickest thing to attempt, and it costs only a few extra words in the prompt.

The technique is not quite as simple as adding one phrase and reading off whatever comes back. The paper’s actual method runs in two stages. The first prompt appends “Let’s think step by step” and lets the model generate its reasoning. A second prompt then feeds that reasoning back in along with a short trigger phrase like “The answer is” to pull out a clean, final answer, separate from the reasoning that produced it.^[1] In an ordinary chat, skipping that second step still works, because a person can read the reasoning and pick out the answer themselves. Any system trying to parse the result automatically needs that second extraction pass to get a clean value out.

The catch is that the model still produces a longer answer, which takes more time and more tokens. The next pages cover models that do this kind of reasoning by default, and how to judge when the extra steps are worth it.