Seed

This phase has quietly relied on randomness at nearly every turn without naming it directly: weights start out “essentially random,” as the Parameters article put it; batches get reshuffled between epochs, as the Epoch article mentioned. All of that randomness is not actually random in the way a dice roll is — it’s controlled by a single number called a seed.

The simple definition

A seed is a starting number that determines the exact sequence of “random” values a computer will generate. Computers can’t produce truly random numbers on demand — what they actually use is a pseudo-random number generator: a mathematical formula that produces a long sequence of numbers that looks random and passes statistical tests for randomness, but is entirely determined by wherever that sequence started. The seed is that starting point. Give the generator the same seed twice, and it will produce the exact same sequence of “random” numbers both times — not similar, but identical, down to the last decimal.

Making a random sequence repeatable

Computers often generate pseudo-random numbers from a starting value called a seed.

import random

random.seed(42)
print(random.randint(1, 100))

Running the same code with the same seed normally begins the same pseudo-random sequence.

Where randomness enters Machine Learning

• Shuffling dataset rows
• Splitting training and test data
• Initializing model weights
• Selecting random augmentations
• Applying dropout
• Sampling generated output

same code + same data + same settings + same seed
                    ↓
           more reproducible experiment

Same seed does not guarantee perfect identity

GPU operations, parallel execution, library versions, hardware, nondeterministic algorithms, and data-loading order can still change results.

A reproducible experiment records:

• Seed values
• Dataset and split version
• Code and dependency versions
• Model and checkpoint version
• Hardware and relevant deterministic settings
• Hyperparameters and preprocessing configuration

Seeds are not model quality

Trying many seeds and reporting only the best result can give a misleading picture. When randomness materially affects results, teams should run several seeds and report the distribution or stability.

Why training needs randomness at all, and why that’s a problem for reproducibility

Recall from the Parameters article that a model’s weights start out essentially random before training begins — this isn’t incidental, it’s necessary: starting every weight at the same fixed value (like zero) would cause a serious, well-known problem where different parts of a network end up learning identical, redundant things, since they’d all start from an identical position and receive identical updates.

Random initialization breaks that symmetry, giving each weight a genuinely different starting point to learn from. Randomness also shows up in batch shuffling between epochs, as mentioned in the Epoch article, and in several of the training techniques covered in the upcoming Generalization phase.

This creates a real, practical problem: if every one of these random choices is genuinely unpredictable, then training the exact same model on the exact same data, twice, could produce two meaningfully different final models — making it hard to know whether a change in results came from an actual improvement, or just random luck in how the weights happened to start out. Setting a seed solves this: it makes the “randomness” itself fixed and repeatable, so re-running the exact same setup produces the exact same result.

flowchart LR
    A[Same seed] --> B[Same weight initialization]
    A --> C[Same batch shuffling order]
    B --> D[Fully reproducible training run]
    C --> D

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of a shuffled deck of cards, shuffled using a specific, repeatable shuffling machine rather than human hands. If you tell the machine “start from position 47 in your shuffling pattern” every time, it will produce the exact same shuffled order every single time you use that starting position — even though the resulting order looks just as jumbled and unpredictable as a genuinely random shuffle would.

Where this breaks down: A physical shuffling machine still involves real mechanical variation between uses. A computer’s pseudo-random generator, given the same seed, produces mathematically identical output every time, with zero variation whatsoever — a level of exact repeatability no physical process can truly match.

Why reproducibility matters this much in practice

Being able to exactly reproduce a training run isn’t just a tidiness preference — it’s essential for real scientific and engineering work.

If a researcher wants to know whether a new technique genuinely improved a model, or whether an unlucky random weight initialization made the baseline look worse than it really was, they need to be able to control for randomness directly — running both the old and new approach with the same seed, so any difference in outcome can be attributed to the actual change being tested, not to random luck.

This is precisely why serious ML research papers commonly report results averaged across multiple different seeds, rather than a single run, and why debugging a training run that behaved strangely often starts with “can we reproduce this exact behavior again” — a question only answerable if the seed was recorded.

Where seeds show up beyond training itself

Seeds aren’t only relevant during training — they matter at inference too, and this is a genuinely practical, current example. Recall from the Prediction article that a language model selects its next token based on a probability distribution, sometimes sampling somewhat randomly rather than always picking the single most likely option (controlled by the temperature setting described there).

OpenAI’s API, along with several other providers, offers a seed parameter specifically for this reason — letting a developer request the same, or very similar, output for the same prompt across repeated calls, which matters for things like automated testing or debugging a chatbot’s behavior.

Worth noting honestly: OpenAI’s own documentation describes this as a “best effort” toward determinism rather than an absolute guarantee, since factors beyond the seed alone (like underlying model or infrastructure updates) can still occasionally cause slightly different output even with the same seed and prompt.

A concrete example, layered

For a simple beginner example: training the one-weight house model twice with seed 42 both times will produce the exact same initial random value for w, the exact same order of training examples, and — assuming nothing else changes — the exact same final trained value, down to the last digit; training it a third time with seed 7 instead will likely produce a slightly different final result, even though the model, data, and every hyperparameter stayed identical.

For a production example: ML research teams training experimental variants of a model architecture will often run each variant across three or five different seeds and report the average performance, specifically to avoid drawing a false conclusion — like “this new technique improved accuracy” — when the real explanation might simply be that one particular seed happened to produce a slightly luckier starting point.

Check your understanding

Does seed 42 have special mathematical quality? No. It is simply a commonly used example.

Can a seed make bad training data good? No. It controls randomness, not data quality or correctness.

Common misconception

A common assumption: that setting a seed makes a model “less random” or somehow worse, since true randomness sounds like it should be more thorough or more genuine. This misunderstands the purpose entirely — the seed doesn’t reduce the amount of randomness used, it only makes that same randomness reproducible on demand.

A model trained with a fixed seed uses just as much genuinely useful randomness (random initialization, shuffled batches) as one without a fixed seed; the only difference is whether you can exactly recreate that specific run again later, which is purely a matter of scientific and engineering discipline, not a trade-off with model quality.

Training seed and sampling seed

A training seed can influence initial weights, data order, dropout masks, and augmentation choices. An inference sampling seed can make a probabilistic decoding experiment easier to repeat when the provider and implementation support it.

The same seed does not promise identical results across different GPU types, library versions, parallel worker counts, or nondeterministic operations. Reproducibility therefore also requires recording code, data versions, hardware, configuration, and checkpoint identifiers.

Closing out this phase

This article completes the Training Mechanics phase, and it’s worth looking back at the full mechanical picture assembled across it: Parameters, made of Weights and Bias, get tuned according to Hyperparameters chosen in advance; a Loss Function measures error; Backpropagation calculates the Gradient for every parameter; Gradient Descent and modern Optimization techniques like Adam use those gradients to update weights, governed by the Learning Rate, working through Batches of a chosen Batch Size, repeated across many Epochs, with Checkpoints protecting progress along the way — all of it controllable and reproducible thanks to a seed.

From here, the glossary moves into a new and closely related question: not just how a model learns, but how well what it learns actually holds up on data it’s never seen — the subject of the upcoming Generalization phase.

In one sentence

A seed is the starting number that makes a computer’s otherwise-unpredictable randomness fully reproducible, and it’s the small, easily-overlooked detail that turns “we trained a model and it worked” into a genuine, repeatable, trustworthy scientific result.

Related Terms

• Parameters
• Training
• Epoch
• Prediction
• Checkpoint