Temperature

The Sampling article, back in the Language Models phase, introduced temperature as part of explaining how a model picks a token from its probability distribution. This article gives it the dedicated, practical treatment it deserves as a real, everyday tool.

The simple definition

Temperature is a setting, typically ranging from 0 to 2, that controls how much randomness goes into a model’s token selection — low values make output more focused and predictable, high values make it more varied and surprising.

Recall from the Sampling article’s mechanical explanation: temperature reshapes the probability distribution before a token gets sampled, sharpening it at low values (the top choice becomes even more dominant) and flattening it at high values (lower-probability tokens get a more meaningful chance).

Why this is the one setting almost everyone actually touches

Recall from the Sampling article that several sampling strategies exist — greedy decoding, temperature, top-k, top-p.

Of all of these, temperature is by far the most commonly exposed and adjusted setting in real products and developer tools, precisely because its effect is intuitive and immediately noticeable: turn it down for consistency, turn it up for variety.

This is exactly why it’s worth a dedicated, practical article here — not just as a mechanical detail, but as a genuine, everyday knob developers and users reach for constantly.

flowchart LR
    A[Temperature near 0] --> B[Focused, consistent, sometimes repetitive]
    C[Temperature around 0.7-1.0] --> D[Balanced, natural variety]
    E[Temperature above 1.5] --> F[Highly varied, occasionally incoherent]

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of asking a knowledgeable friend the same trivia question repeatedly. At “low temperature,” they always give the single answer they’re most confident about, every time, word for word.

At “high temperature,” they start volunteering different, more adventurous answers each time you ask — sometimes genuinely insightful alternatives, sometimes wildly off-base guesses, since they’re now willing to voice answers they’re much less sure about.

Where this breaks down: A friend’s varying confidence reflects genuine uncertainty and personality.

Temperature is a precise mathematical adjustment applied to the softmax calculation covered in the Probability Distribution article — reshaping the exact same underlying distribution the same way every single time, with no personality or genuine uncertainty involved, just a consistent mathematical transformation of probabilities.

Choosing the right temperature for the actual task at hand

This is worth being concrete about, since the “right” temperature genuinely depends on what you’re using the model for, and it’s a real, practical decision developers make constantly.

A low temperature (often 0 to 0.3) suits tasks needing consistency and precision — extracting structured data from a document, answering factual questions, writing code, or any task where you want the same input to reliably produce very similar output.

A moderate temperature (often 0.7 to 1.0) suits general conversational use, balancing coherence with natural variety. A higher temperature (above 1.0) suits creative tasks — brainstorming, poetry, generating varied marketing copy — where surprising, less-obvious word choices are a genuine feature rather than a bug.

A concrete example, layered

For a simple beginner example: asking a model to complete “The opposite of hot is” at temperature 0 will almost always produce “cold,” every single time, since that’s overwhelmingly the most probable continuation; at temperature 1.5, it might occasionally produce something like “cool” or even a more unusual, less predictable word, since the flattened distribution gives those lower-probability options a real, if still modest, chance.

For a production example: OpenAI’s API documentation exposes a temperature parameter directly to developers, and it’s common practice for a company building a code-generation feature to set temperature near 0 (consistency matters far more than creative variety for working code), while a company building a creative-writing assistant might default to a temperature around 0.9 or higher, deliberately embracing more variation as a genuine product feature.

Why very high temperatures can actually hurt

It’s worth being honest about a real, well-documented limitation, not presenting “higher temperature” as a purely creative upside with no cost. Recall from the Probability Distribution article that even low-probability tokens still represent statistically less likely, sometimes outright poor continuations.

At sufficiently high temperatures, a model becomes meaningfully more likely to select these poor options — leading to output that can become incoherent, factually unreliable, or simply strange, well past the point of “creative variety” into genuine quality degradation.

Temperature isn’t a dial where higher always means “more creative” in a purely positive sense — past a certain point, it just means “noisier.”

See temperature reshape one distribution

Assume three next-token logits are 2.0, 1.0, and 0.0. Before softmax, each logit is divided by temperature:

Token	Original logit	At T = 0.5	Approx. probability	At T = 2.0	Approx. probability
blue	2.0	4.0	86.7%	1.0	50.6%
clear	1.0	2.0	11.7%	0.5	30.7%
singing	0.0	0.0	1.6%	0.0	18.6%

Low temperature makes the strongest option dominate. High temperature flattens the distribution, giving weaker choices more opportunity. Temperature does not teach the model new facts or improve weak logits; it reshapes the choice probabilities already produced.

Google’s current Gemini prompting guide recommends retaining default sampling values for Gemini 3.x, warning that changes—including lowering temperature below 1—can cause looping or degraded performance on complex reasoning tasks. Production defaults are model-specific; “always use temperature 0 for accuracy” is not a universal rule.

Real-world example: product codes versus story ideas

A warehouse assistant generating an exact product category benefits from stable output and schema validation. A children’s story-idea generator benefits from some variety so every request does not begin identically.

Task: map “blue running shoe” to a catalog category
Goal: consistency; use provider defaults first and evaluate deterministic settings
Task: suggest five surprising planets for a story
Goal: diversity; a somewhat broader sampling distribution may help

The application sends temperature as a generation setting alongside the prompt. It is not written as an English sentence and it does not enter as a normal token. During every output step, it reshapes logits before sampling.

Current Gemini guidance recommends leaving Gemini 3.x sampling parameters at their model defaults unless testing shows a reason to change them; model families are tuned differently.

Common misconception

A frequent beginner assumption: that temperature controls how “smart” or “capable” a model is, with higher settings unlocking better reasoning.

This isn’t accurate — temperature only affects how a token gets selected from an already-calculated probability distribution, not the model’s underlying knowledge, capability, or the quality of that distribution itself.

A model’s actual competence at a task comes from its training and architecture, entirely separate from this sampling-time setting.

Where this fits in what comes next

You now understand the most commonly adjusted sampling setting in practical use. The next article, Top-k, covers a related but distinct sampling control — one that limits which tokens are even eligible to be considered, rather than reshaping the probabilities of every token the way temperature does.

In one sentence

Temperature controls how much randomness enters a model’s token selection, from focused and predictable at low values to varied and creative at high ones, and choosing the right value — matched to whether the task needs consistency or variety — is one of the most immediately practical, everyday decisions in working with generative AI.

Related Terms

• Sampling
• Probability Distribution
• Top-k
• Top-p
• Prompt Engineering