Algorithm

Say you want to bake a cake. A recipe tells you, step by step, exactly what to do with your ingredients: mix these, in this order, at this temperature, for this long. The recipe itself isn’t the cake — it’s the procedure that turns raw ingredients into a finished cake. In Machine Learning, that same idea has a name: algorithm.

The simple definition

An algorithm is a step-by-step procedure a computer follows to solve a problem or perform a task. This word isn’t unique to AI — it’s one of the oldest and most general terms in all of computer science. Sorting a list of names alphabetically uses an algorithm. Finding the shortest driving route on a map uses an algorithm. Neither of those has anything to do with learning from data.

What makes an algorithm relevant to this glossary specifically is a particular category of them: learning algorithms — the step-by-step procedures that take Features and Labels from a Dataset and use them to actually produce a trained Model, the topic covered right after this one.

Following a simple algorithm

Imagine finding the largest number in a list:

1. Treat the first number as the largest seen so far.
2. Examine the next number.
3. If it is larger, remember it instead.
4. Repeat until the list ends.

[4, 9, 2, 7]
 ↓
largest = 4
compare 9 → largest = 9
compare 2 → keep 9
compare 7 → keep 9
output = 9

Algorithm versus model

Recipe                = algorithm
Cake produced         = result

Learning procedure    = training algorithm
Learned parameters    = trained model

The learning algorithm describes how to learn. The trained model contains what was learned from a particular dataset. Running the same learning algorithm on different datasets can produce different models.

A learning algorithm in motion

Suppose a model predicts house price using:

predicted price = size × weight

flowchart LR
    A[Choose starting weight] --> B[Predict prices]
    B --> C[Measure prediction error]
    C --> D[Change weight to reduce error]
    D --> B
    B --> E[Stop after chosen condition]

The algorithm specifies how predictions, errors, and updates are calculated. The final learned weight belongs to the resulting model.

A non-ML code example

def find_largest(numbers):
    largest = numbers[0]

for number in numbers[1:]:
        if number > largest:
            largest = number

return largest

print(find_largest([4, 9, 2, 7]))  # 9

largest stores the best answer found so far. The loop examines every remaining number, the condition updates the answer when necessary, and the function returns after all values have been checked.

Choosing an algorithm in production

Teams compare algorithms based on the data type and amount, required quality, training and inference speed, memory and compute cost, interpretability, ability to handle changing data, and operational complexity.

A simple algorithm provides a useful baseline. A more complicated method should earn its complexity through measurable improvement that matters to the product.

Algorithm mistakes

• Calling a trained model an algorithm as if the terms were identical
• Assuming a fashionable algorithm fits every dataset
• Comparing methods using different test data
• Ignoring runtime, memory, latency, and maintenance costs
• Optimizing a metric that does not represent the real user outcome
• Forgetting boundary cases, invalid inputs, or stopping conditions

Why this distinction matters: algorithm vs. model

This is one of the most commonly confused pairs of terms in the entire field, so it’s worth being precise here. The algorithm is the procedure. The model is the result of running that procedure on data. They are not the same thing, even though people often use the words loosely and interchangeably in casual conversation.

flowchart LR
    A[Learning Algorithm] --> B[Trained on: Features + Labels]
    B --> C[Trained Model]

Think of it like this: the recipe (algorithm) plus the actual ingredients (data) produces the actual cake (model). You could take the exact same recipe and use it with different ingredients — same algorithm, different dataset — and you’d end up with a different cake. Similarly, the exact same learning algorithm, trained on different data, produces a different model.

ANALOGY vs. TECHNICAL REALITY

Analogy: A recipe (algorithm) tells you the general procedure — combine flour and sugar, bake at 350°F for 30 minutes. Follow that same recipe with chocolate ingredients versus vanilla ingredients, and you get two different cakes (models), even though the underlying procedure never changed.

Where this breaks down: A recipe is fixed and doesn’t adjust itself as you cook. A learning algorithm, in contrast, is specifically built to adapt — its whole purpose is to repeatedly adjust a model’s internal parameters based on how wrong its guesses are, exactly as described in the training loop from the Machine Learning article. The “recipe” here isn’t a fixed sequence of static actions; it’s a mathematical procedure for iterative self-correction.

What a learning algorithm actually specifies

A learning algorithm defines things like: how the model should make an initial guess, how to measure how wrong that guess was (an idea called a loss function), and precisely how to adjust the model’s internal parameters to reduce that wrongness on the next attempt. Different algorithms make different choices about exactly how to do this adjustment, which is where the real variety in the field comes from.

A few well-known learning algorithms, briefly

You don’t need to memorize the mechanics of these — just recognize that “algorithm” refers to a family of different, named procedures, each suited to different kinds of problems:

• Linear regression — a simple algorithm suited to predicting a number from features that have a roughly straight-line relationship to the outcome (like predicting a house price from square footage).
• Decision trees — an algorithm that learns a series of branching yes/no questions about the features to arrive at a prediction, similar in spirit to a flowchart.
• Gradient descent–based algorithms — a broad and extremely important family, used to train the Neural Networks behind most modern Deep Learning systems, which work by repeatedly nudging parameters in the direction that reduces error, exactly as sketched in the Machine Learning article’s training loop.

Choosing the right algorithm for a given problem is itself a skill — some algorithms handle certain kinds of data or relationships far better than others, and part of an ML engineer’s job is knowing which one fits the problem at hand.

Which algorithms power models like GPT and Gemini

It’s worth naming the actual algorithm behind today’s most talked-about systems, because it demystifies a lot of the mystery around them. Models like OpenAI’s GPT and Google’s Gemini are both built on the same core architecture — the Transformer, introduced by Google researchers in 2017 — trained using gradient descent and a supervised-style prediction task: given the words seen so far, predict the next one, over and over, across enormous amounts of text.

The Transformer’s key algorithmic idea is self-attention — a mechanism that lets the model weigh how relevant every other word in a sentence is to the word it’s currently processing, regardless of how far apart they are. That’s what lets these models handle long, context-dependent sentences well.

On top of that shared foundation, each lab layers its own engineering choices: GPT-4, by some public reporting, isn’t a single massive model at all but several smaller expert models working together behind the scenes (an approach called “mixture-of-experts”). Gemini, meanwhile, is built on a Transformer decoder specifically tuned for stable training at large scale and fast inference on Google’s own custom hardware, using efficiency-focused attention variants to handle long inputs well.

So when people say “GPT uses a different algorithm than Gemini,” that’s not quite accurate — both are Transformer-based, trained with gradient descent; the real differences are in scale, data, and engineering refinements layered on top of that shared algorithmic core.

How this actually plays out inside a company

If you’re picturing an engineer at a company inventing a brand-new algorithm from scratch for every project, that’s the exception, not the rule. In the overwhelming majority of real-world ML work, engineers use existing, well-established algorithms rather than writing new ones — the same way a web developer doesn’t reinvent how a database sorts rows, they just call a sorting function that already exists and is known to work.

In practice, this usually looks like:

• Using a library that already implements the algorithm. Tools like scikit-learn (for classic algorithms like decision trees and linear regression) or PyTorch and TensorFlow (for building neural networks and Transformers) provide these algorithms as ready-to-use building blocks. An engineer typically writes code that configures and calls these implementations rather than coding the underlying math by hand.
• Starting from an existing pretrained model rather than training one from nothing. Platforms like Hugging Face host thousands of already-trained models — including open ones like Meta’s Llama — that a company can download and adapt to their own data, a much cheaper and faster path than building a Transformer-based model from scratch, which realistically only a handful of well-resourced labs (OpenAI, Google, Anthropic, Meta, and a few others) currently do at the frontier scale.
• Genuinely inventing a new algorithm is rare, and mostly happens in dedicated research teams or academia — it requires deep expertise, extensive experimentation, and is usually published in a paper (exactly how the Transformer itself became public knowledge in 2017) before the rest of the industry adopts it.

So, to directly answer “are these algorithms available, or do we need to write them”: for the vast majority of practical, real-world work, the algorithms already exist and are freely available as open-source code or research papers. What a company actually builds in-house is usually the surrounding system — which existing algorithm to use, how to prepare the data, how to configure and train it, and how to deploy the resulting model — not the core mathematical algorithm itself.

A concrete, end-to-end example

Take the hospital readmission example from the Label article. An engineer might choose a decision-tree-based algorithm because it tends to produce results that are relatively easy for doctors to interpret and trust (“readmission predicted because: age over 65, and length of stay under 2 days, and no follow-up scheduled”). They’d feed that algorithm the training set’s features and labels. The algorithm’s job, at that point, is to run its specific procedure over that data and produce a trained model — a decision tree with specific branching thresholds, tuned to that particular hospital’s actual historical data.

Key terms

• Algorithm: A defined sequence or method for completing a task.
• Learning algorithm: A method that adjusts a model using data.
• Model: The learned structure and parameter values used for predictions.
• Iteration: One repetition of an algorithmic process.
• Complexity: The time, memory, or operational resources a method requires.

Check your understanding

Are all algorithms AI? No. Most ordinary software also relies on algorithms.

Are algorithm and model interchangeable words? No. A learning algorithm produces or adjusts a model; the model is later used for inference.

Common misconception

People often say “the algorithm predicted X,” when what they usually mean is “the model predicted X.” Once training is finished, the algorithm’s job is done — it doesn’t run again during everyday predictions. It’s the resulting trained model, not the algorithm itself, that’s actually deployed and used to make predictions on new data. Keeping this distinction straight — algorithm builds the model, model makes the predictions — will make the next few articles in this sequence much easier to follow.

Architecture, training algorithm, and trained model

An architecture is the arrangement of calculations. A training algorithm is the procedure that adjusts learned values. A trained model is the architecture together with its learned values after training.

GPT and Gemini use Transformer-based architectures. Self-attention is one important calculation inside that architecture. Backpropagation and an optimizer help train the network. The finished learned parameter values form a particular trained model.

Where this fits in what comes next

You now understand the procedure. The next article, Model, covers the actual result of running an algorithm on data — the thing that gets deployed, used, and interacted with in a real system. After that, Training zooms into the process by which the algorithm and dataset actually come together to produce that model, filling in the mechanical detail this article has only sketched so far.

In one sentence

An algorithm is the step-by-step learning procedure — not the end result — and understanding that a fixed algorithm produces a different model depending on the data it’s trained on is the key to keeping this whole part of the vocabulary straight.

Related Terms

• Feature
• Label
• Dataset
• Machine Learning (ML)
• Model