The Few-Shot article covered showing a model examples of final answers. This article covers a related but genuinely distinct technique — showing, or eliciting, the reasoning that leads to an answer, not just the answer itself: chain-of-thought prompting.
The simple definition
Chain-of-thought prompting means encouraging a model to work through a problem step by step, explicitly generating intermediate reasoning before arriving at a final answer, rather than jumping straight to a conclusion. Recall from the Next-Token Prediction article that a model generates one token at a time, each one building on everything generated so far.
Chain-of-thought exploits this directly — by generating explicit intermediate steps as actual tokens in the sequence, the model gives its later tokens genuine, concrete reasoning to build on, rather than having to arrive at a complex answer in one single, unsupported leap.
Why writing out the steps genuinely helps, mechanically
This connects precisely to the autoregressive structure covered in the Autoregressive Generation article. When a model is asked a multi-step math problem and told to answer immediately with just a number, it has to somehow compress an entire multi-step calculation into the probability distribution for one single token — a genuinely hard demand.
When it’s instead encouraged to write out “First, I’ll calculate X. Then, using that result, I’ll calculate Y…” each of those intermediate steps becomes real tokens in the sequence, which the model can then attend to — as covered in the Self-Attention article — when generating the next step.
The model isn’t secretly reasoning invisibly and then just being asked to explain afterward — the explicit, written-out steps are doing genuine computational work, giving later predictions concrete intermediate results to build from.
flowchart LR
A[Direct answer: jump straight to final output] --> B[Harder for complex, multi-step problems]
C["Chain-of-thought: 'First... then... therefore...'"] --> D[Each step becomes real context for the next]
D --> E[More reliable final answer on complex problems]ANALOGY vs. TECHNICAL REALITY
Analogy: Think of solving a multi-step word problem in your head versus on paper.
Trying to hold every intermediate calculation in your head simultaneously is genuinely harder and more error-prone than writing each step down as you go, then using what’s already written to figure out the next step.
Writing it down doesn’t just document your thinking — it actually makes the thinking itself more reliable.
Where this breaks down: A person writing on paper has a stable, external memory that doesn’t degrade.
A model’s “written” reasoning is simply more tokens added to its own sequence, which then get processed through the exact same next-token prediction mechanism as everything else — the improvement isn’t from an external memory aid in the human sense, it’s from giving the model’s own attention mechanism concrete, explicit intermediate results to condition on, rather than requiring it to hold an unstated calculation implicitly.
The real, published research behind this technique
This is worth grounding precisely, since chain-of-thought has a specific, well-documented research origin.
A 2022 paper by Wei and colleagues, titled “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” demonstrated that simply prompting a model to show its work — often using few-shot examples that included worked-out reasoning steps, not just final answers — produced substantial accuracy improvements on math and logical reasoning tasks.
A separate, related 2022 paper by Kojima and colleagues, “Large Language Models are Zero-Shot Reasoners,” found something even simpler and more striking: just adding the specific phrase “Let’s think step by step” to a prompt — with no worked examples at all — meaningfully improved reasoning performance on its own.
A concrete example, layered
For a simple beginner example: asked “If a train travels 60 miles in 2 hours, then 90 miles in 3 hours, what’s its average speed for the whole trip?” a model prompted to answer directly might make an arithmetic slip; the same model prompted with “let’s think step by step” might write “Total distance: 60 + 90 = 150 miles.
Total time: 2 + 3 = 5 hours. Average speed: 150 / 5 = 30 mph” — each step providing a concrete, correct intermediate result for the next.
For a production example: this specific technique — eliciting step-by-step reasoning before a final answer — became foundational to the reasoning-focused models covered later in this phase, including OpenAI’s o1 and Anthropic’s extended-thinking Claude models, both of which build extensive, automatic step-by-step reasoning directly into how they generate a response, rather than requiring a user to explicitly request it through prompting.
Why chain-of-thought isn’t free
It’s worth being honest about a real, practical trade-off, directly connecting to the Token and Context Length articles’ cost discussion.
Generating explicit reasoning steps means generating meaningfully more tokens per response, which directly increases both the time and the cost of getting an answer — a real, measurable trade-off between reliability and speed/cost that becomes especially significant for the reasoning models covered later in this phase, which can generate substantially more reasoning tokens than a typical direct response.
Ask for a useful answer structure, not private internal text
For many applications, the user needs a concise, checkable explanation—not every hidden intermediate token a reasoning model may generate.
Useful request:
Solve the problem. Return:
1. the formula used,
2. the important substituted values,
3. the final answer with units,
4. one quick verification.For distance = speed × time, with 60 km/h for 2.5 hours, the visible explanation can show 60 × 2.5 = 150 km. That is enough to teach and verify the solution. A model’s private reasoning process can be longer, uncertain, or represented internally in a way that is not identical to this polished explanation.
Chain-of-thought prompting, model-generated reasoning tokens, and a user-facing explanation are related but not identical:
| Term | Meaning |
|---|---|
| Chain-of-thought prompt | A prompting technique requesting intermediate reasoning. |
| Reasoning/thinking tokens | Model work performed before or around the final answer in supported reasoning models. |
| Explanation | The concise reasoning evidence intentionally shown to the user. |
Real-world example: checking a delivery calculation
A route-planning assistant must determine whether a van can finish three trips before its battery reaches the reserve limit. Asking only for an answer can hide an incorrect assumption. The application can request a short, checkable calculation:
Show the distance used for each trip, add them, compare the total with
the usable battery range, and state the remaining range.The visible response might show 42 + 36 + 28 = 106 km and 140 − 106 = 34 km remaining. A calculator tool can verify the arithmetic.
Current reasoning models from GPT and Gemini may also perform private thinking before producing that explanation; the private thinking and the concise evidence shown to the user are not necessarily the same text.
Common misconception
A frequent beginner assumption: that chain-of-thought reasoning shown by a model is simply an after-the-fact explanation, generated for the user’s benefit, with the model having already “known” the answer some other way.
As the mechanical explanation above showed, this isn’t accurate — the written-out steps are genuinely part of the computation itself, functioning as real intermediate context the model’s later predictions actually depend on, not a decorative narration layered on top of an answer arrived at through some separate, hidden process.
Where this fits in what comes next
You now understand how eliciting explicit reasoning steps genuinely improves a model’s reliability on complex tasks.
The next three articles, Temperature, Top-k, and Top-p, shift focus to a different lever entirely — the sampling settings, first introduced back in the Sampling article, that control how a model actually selects each of those tokens, reasoning steps included.
In one sentence
Chain-of-thought prompting encourages a model to generate explicit, step-by-step reasoning before its final answer, and because each written step becomes real context the model’s next prediction can build on, this simple technique produces genuine, published, sometimes dramatic accuracy improvements on complex, multi-step problems.
Related Terms
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed