Introduction to AI

Imagine asking a computer for help in your own words.

You do not need to learn a special command. You do not need to fill in a complicated form. You simply describe what you want, and the computer tries to understand your request.

For example, it might:

• rewrite a messy paragraph so it is easier to read;
• look at the food in your half-empty fridge and suggest a meal; or
• read a programming error and point to the line that caused the problem.

These abilities feel normal today, but they were not always possible. For most of computing history, computers could only follow instructions written by people.

They were fast.

They were reliable.

But they did not understand what people meant.

So what changed?

There was no single invention and no sudden moment when computers became intelligent. Instead, researchers developed one idea after another. Each new idea tried to solve a limitation in the idea before it.

This tutorial follows that story in a simple order. We will start by asking what intelligence means, then see why traditional programs were limited, and finally connect those ideas to the AI systems people use today.

You do not need to know Python or write code before starting. Every idea begins with an everyday example and gradually moves toward the technical explanation. If you already build software, the examples may be familiar, but the way they connect will give you a useful map of modern AI.

Here is the visual roadmap of the journey we will take in this tutorial, showing how each concept evolved into the next:

graph TD
    A[Rule-Based / Expert Systems] -->|Learn patterns from examples| B[Machine Learning]
    B -->|Use neural networks with many layers| C[Deep Learning]
    C -->|Process sequences with attention| D[Transformers]
    D -->|Train language models at large scale| E[Large Language Models]
    E -->|Post-train for instructions and safety| F[Chat Assistants]
    F -->|Add tools, state, and an action loop| G[Agentic AI]

    style A fill:#f9f,stroke:#333
    style E fill:#bbf,stroke:#333
    style G fill:#bfb,stroke:#333

A Short History Without the Hype

AI did not begin with chatbots, and progress did not move upward in a smooth line.

Period	What changed	Why it mattered
1950	Alan Turing published “Computing Machinery and Intelligence”	It turned a philosophical question into something people could examine through behavior
1955–1956	The Dartmouth proposal used the name “artificial intelligence”	It helped give the research field a shared name
1960s–1980s	Researchers built search, symbolic, and expert systems	Machines solved narrow problems, but hand-maintained knowledge was difficult to scale
AI winters	Funding and enthusiasm fell when results did not match large promises	Impressive demonstrations were not yet reliable general systems
1990s–2010s	Statistical ML, larger datasets, GPUs, and improved training grew	More systems learned patterns from examples
2017	Transformers made attention-based sequence training more parallelizable	The architecture became the foundation for most widely used LLMs
2020s	Generative and multimodal models reached products and APIs	AI entered everyday writing, coding, image, audio, and tool-using applications

This is a map, not a list of sudden inventions. Older ideas continued to matter, and each milestone depended on earlier work.

What Are We Even Trying to Build?

Before chasing “artificial” intelligence, it’s worth pausing on the word “intelligence” itself — because it’s slipperier than it sounds.

💡 Think about it: Is a calculator intelligent? It can multiply nine-digit numbers faster than any human alive. Yet nobody calls it smart. Why not?

The honest answer is that intelligence isn’t one skill — it’s a bundle of different skills: recognizing patterns, using language, planning ahead, adapting when things don’t go as expected, drawing on past experience to handle something brand new. A calculator has exactly one of those (arithmetic), applied with zero flexibility. That’s not intelligence — that’s a very fast, very narrow tool.

💡 Think about it

A GPS can calculate a route with a fixed search algorithm. It may also use machine learning to predict traffic. One product can therefore contain both ordinary algorithms and AI.

A self-driving system combines programmed safety rules, maps, sensors, and learned models. Real products are often mixtures rather than simple “AI” or “not AI” boxes.

So here’s a working definition to carry through this whole tutorial:

Artificial Intelligence is the attempt to build machines that can perform tasks which, if a person did them, we’d call “intelligent” — tasks that require judgment, adaptation, or understanding, not just speed.

Notice what that definition doesn’t say. It doesn’t say how the machine has to do it. That is deliberate because AI is a broad umbrella. It can include expert rules, machine learning, planning, robotics, and generative models.

The boundary is not perfectly agreed upon. Ask a clearer question: Which part follows fixed instructions, and which part learns or makes uncertain predictions?

AI, Machine Learning, and Deep Learning Are Not Synonyms

Artificial Intelligence
├── Rule-based and search systems
└── Machine Learning
    ├── Decision trees and other ML methods
    └── Deep Learning
        ├── Image and speech models
        └── Many language and multimodal models

• AI is the broad goal and field.
• Machine Learning is one approach within AI that learns patterns from data or experience.
• Deep Learning is one family of ML methods built with multilayer neural networks.

A system can be AI without machine learning, and it can use machine learning without being a chatbot.

Production reality check: when not to use AI

Prefer a simple rule when it solves the problem completely and safely.

• Predictability: A deterministic rule gives the same result for the same relevant input. Ordinary programs can still contain bugs or depend on changing data.
• Cost and speed: A small rule is usually cheaper and faster than a large model, although the exact difference depends on the task.
• Testing: A fixed rule usually has a clearer expected result. A probabilistic model needs evaluation across many examples and monitoring after release.

Always start by asking: Can I solve this with a simple database query, a regex pattern, or basic heuristics? If yes, do not use AI.

Why Machines Needed Rules Before They Needed Judgment

To see why “judgment” was ever a problem, you need to understand what a computer could do before anyone tried to make it smart.

Picture a vending machine. You press B4, it drops a bag of chips. Press C2, it drops a soda. It doesn’t “decide” anything — it matches an input to a fixed output, the same way, every single time. It has no idea what hunger is, no opinion about snacks, no memory of what you bought yesterday. It just executes a wiring diagram someone designed in advance.

Scale that idea up by a few billion, and you basically have a traditional computer program. Every “if this happens, do that” is a tiny vending-machine decision, and a program is millions of those decisions wired together with extraordinary precision.

TRADITIONAL PROGRAM
---------------------
INPUT ---> [ RULES WRITTEN BY A HUMAN ] ---> OUTPUT

Example: Shipping cost calculator
INPUT: weight = 8kg, destination = "international"
  |
  v
RULE: if weight <= 1kg and domestic: cost = $5
      elif weight <= 5kg and domestic: cost = $9
      elif international: cost = $25 + $2 per kg over 2kg
  |
  v
OUTPUT: cost = $37

To visualize how the paradigm shifted:

graph TD
    subgraph Traditional Programming
        I[Input Data] --> P[Programmer's Rules]
        P --> O[Output Answer]
    end

    subgraph Machine Learning
        I2[Input Data] --> M[ML Training]
        A2[Target Answers] --> M
        M --> R[Trained Model / Rules]
    end

    style P fill:#f9f,stroke:#333,stroke-width:2px
    style M fill:#bbf,stroke:#333,stroke-width:2px

To see the difference in adaptability, compare a Vending Machine (traditional programming) with a Personal Chef (machine learning). A vending machine is completely rigid: B4 always drops chips, even if you are diabetic and need sugar-free snacks. A personal chef learns your tastes over time, adjusts their recipes based on what ingredients are fresh in the market, and adapts when you tell them a dish was too salty. The chef doesn’t follow a hardcoded wiring diagram; they use experience to apply judgment.

This is not a lesser or outdated way to build software — for problems like this, it’s still the correct one. Nobody should replace a shipping calculator, a payroll system, or an ATM’s withdrawal logic with a neural network. The rules are fixed, finite, and fully known in advance, so writing them down by hand is fast, cheap, and perfectly predictable.

For the experienced folks in the room: this is the same reason you don’t reach for machine learning to validate an email address or enforce a business rule you can express in one if statement. AI is not a universal upgrade to software — it’s a specific answer to a specific kind of problem, which we’re about to define precisely.

So Where Does It Break?

Try writing explicit rules for these instead:

• “Is this text message spam?”
• “Does this X-ray show a tumor?”
• “Is this customer likely to cancel their subscription next month?”
• “Does this sentence sound sarcastic?”

Sit with the spam one for a second. You could start listing rules — “block if it contains the word ‘winner’” — but spammers adapt within hours. You could add “unless it’s from a known contact” — now you need a rule for that too. Every rule you add creates new edge cases, and real language has effectively infinite ways to say the same thing. There’s no bottom to this list.

🚀 The turning point Rule-based software works best when people can describe the important cases clearly and maintain the rules as the world changes. Many valuable problems do not fit that shape because useful patterns are too numerous, fuzzy, or difficult to describe completely.

That gap — what do we do when nobody can write down all the rules, but the machine still needs to make a useful prediction? — is one major reason machine learning became important. AI is broader and also studies search, planning, reasoning, robotics, and other ways of producing intelligent behavior.

The First Attempt: Teach the Machine Your Knowledge Directly

If a human expert makes good decisions by mentally following a set of rules, the earliest serious attempt at “AI” made an almost embarrassingly literal move: interview the expert, extract their rules, and type them into a program by hand. Thousands of them, if that’s what it took.

This approach — often called a knowledge-based or rule-based system — genuinely worked, within limits. Early systems built this way could diagnose specific diseases, configure complex orders, and hold surprisingly convincing scripted conversations, decades before anything we’d recognize as modern AI existed.

But push this idea hard enough, and it starts to strain under its own weight:

Problem	Why it happens
Too many possible situations	Real situations combine in too many ways to write a rule for each one
Brittleness	A slightly unexpected input can confuse the system
No self-improvement	Every fix requires a human to notice the gap and hand-write a patch
The knowledge acquisition bottleneck	Experts often can’t fully explain their own intuition as clean rules

That last row is the deepest one, and it’s worth slowing down on. An experienced radiologist can glance at a scan and sense something is off before they can point to exactly why. A fluent speaker can tell a sentence sounds “off” without citing the grammar rule it breaks. That kind of knowledge is real — genuinely valuable — but it was never built out of explicit rules in the first place. You can’t extract a rule that was never consciously formed. And that meant no amount of interviewing, no matter how thorough, could ever fully capture it.

🚗 The Driving Rules Analogy: Imagine trying to write a manual of rules on how to drive a car:

• Rule 1201: If a ball rolls into the street, press the brakes with 30% force.
• Rule 1202: If the ball is red, press brakes with 35% force.
• Rule 1203: If a dog is chasing the ball, swerve slightly to the left, unless there is a mailbox on the left… It is immediately obvious that you could write a million rules and still crash on your first drive because real-world driving is based on continuous, sub-conscious spatial intuition, not a checklist.

This is the wall that quietly ended the “hand-write all the intelligence” era, and it’s what forced researchers to ask a very different question:

“What if, instead of telling the machine the rules, we just showed it enough examples — and let it work out the pattern on its own?”

This question is where the story actually starts to accelerate.

Production Context: Are Expert Systems Dead? Rule-based expert systems did not simply disappear when machine learning grew. Business rules and decision tables remain useful for tax calculations, compliance checks, and billing rules. When a company must show exactly which declared rule caused a decision, an explicit rule system may be easier to audit than a learned model.

The Shift That Changed Everything: Machine Learning

Here’s the idea, before the label. Suppose you want a computer to recognize whether a photo contains a cat. You could try to hand-write rules — pointy ears, whiskers, a certain silhouette — but cats come in endless poses, breeds, angles, and lighting. There’s no finite rulebook that covers them all.

So try something else entirely. Show the computer thousands of photos, each one labeled “cat” or “not cat.” Let it compare them, notice which patterns of pixels tend to show up in the “cat” photos and not in the others, and gradually adjust itself until it gets better and better at guessing correctly on photos it’s never seen before.

Nobody wrote the rule “pointy ears + whiskers = cat.” The machine built its own internal sense of “cat-ness,” purely from examples. That’s Machine Learning (ML):

Instead of a programmer hand-coding the logic, the system learns the logic — automatically — from data.

This shift reduces one part of the knowledge-acquisition problem, but it does not make the problem disappear. People still decide what outcome matters, collect suitable data, check its quality, choose a model, evaluate failures, and decide whether the system is safe enough.

The cat example uses supervised learning because each photo has a label. Not all machine learning works this way:

Learning setup	Available feedback	Simple example
Supervised learning	Examples include target answers	Photos labeled “cat” or “not cat”
Self-supervised learning	Data creates its own target	Predict the next token in existing text
Unsupervised learning	No target label is supplied	Group similar customers
Reinforcement learning	Actions receive rewards or penalties	Learn a game strategy from scores

Modern systems can combine these approaches.

Before going further, let’s name the main pieces:

• A model is the part of the system that turns an input into a prediction or another output.
• A weight is a number inside the model that controls how strongly one signal influences another.
• A parameter is a learned number inside a model. In a neural network, most parameters are weights.
• Training is the process of changing those numbers so the model makes better predictions.
• Inference is using the trained model to make a prediction on new input. During inference, the model normally uses its learned numbers without changing them.

You can think of a model as a collection of adjustable numbers that has learned a useful pattern. It is not a human mind stored inside a file, and it is not a list of rules that someone wrote one by one.

TRADITIONAL PROGRAMMING          MACHINE LEARNING
--------------------------      --------------------------
 rules + data --> answers        data + answers --> rules
 (human writes the logic)        (machine discovers the logic)

That flipped arrow is, honestly, the single most important idea in this entire tutorial. Everything from here forward is a variation on it.

🎛️ The Soundboard Sliders Analogy: To make weights and parameters concrete, imagine a massive sound mixing board in a recording studio with millions of sliders:

• Weights / Parameters: Each slider controls the volume or tone of a single instrument’s signal. The position of each slider is a weight.
• Training: You play a raw song through the board. The output sounds terrible. The producer (the training algorithm) compares it to how the song should sound and slightly nudges some sliders up and others down. You repeat this for hundreds of tracks until the board is perfectly tuned.
• Inference: Once the board is mixed, you tape the sliders down so they cannot move. You play a new, unheard song through the locked board. It instantly sounds great because the slider positions (weights) have captured the general rules of a good mix.

But How Does a Machine “Learn” Anything?

Here’s the intuition, stripped down to its simplest version.

🧳 The Luggage Weight Analogy: Imagine trying to guess the weight of a suitcase just by lifting it. At first, you guess blindly. A friend tells you if your guess is “too high” or “too low.” Each time you get feedback, you adjust your internal muscle memory. After lifting 100 bags, you can guess the weight of a new bag with high accuracy without ever looking at a scale. That muscle adjustment is training; guessing the weight of a new bag is inference.

In this supervised-learning example, the model makes a prediction, compares it with the target answer, measures the error, and adjusts its internal settings. Repeating this across many examples can improve performance on new data. Improvement is not guaranteed: poor data, a weak objective, an unsuitable model, or overfitting can still produce a bad result.

Here is the same idea with a small example. Imagine a model looking at a picture and answering, “How likely is this to be a cat?”

1. The model sees a picture and predicts: 40% cat
2. The correct label says: cat
3. The prediction is too low, so the model measures an error
4. Training adjusts many weights by tiny amounts
5. The model tries another picture
6. After many examples, its predictions usually become more accurate

The model does not improve because somebody edits a rule such as “cats have whiskers.” It improves because repeated corrections change the numbers inside it. The examples used while those numbers are changing belong to the training phase. Later, when the model receives a new picture and produces an answer without changing its weights, that is inference.

Production engineering: training and inference

Training and inference have different jobs and are often separated operationally.

• Training can range from a laptop experiment to a large accelerator cluster, depending on the model.
• Inference may run on CPUs, GPUs, phones, browsers, or specialized chips. The acceptable time could be milliseconds for autocomplete or hours for an offline batch job.
• Data work may include permission checks, cleaning, labeling, deduplication, versioning, and quality evaluation. Labeling is crucial for some projects; self-supervised projects rely more on filtering and curation.

✅ Key takeaway Machine Learning didn’t just add a new tool to the toolbox — it changed the entire type of question engineers could ask. Instead of “how do I encode this knowledge?” the question became “how do I collect enough good examples?”

Checkpoint: You should now be able to explain that a model is a pattern-finding system, weights are adjustable numbers, training changes those numbers, and inference uses the trained model on new input.

Teaching Machines to Learn Like (a Simplified Version of) a Brain

Early machine learning models were often fairly simple mathematical functions, good at finding straightforward patterns but limited when problems got messy — say, telling apart ten different handwritten digits, or recognizing a face across different lighting.

Researchers borrowed loose inspiration from biology. This analogy is useful as a first picture, but an artificial “neuron” is only a mathematical calculation; it is not a realistic copy of a brain cell.

This inspired the artificial neural network — not a literal brain simulation, but a mathematical structure loosely modeled on the same idea: layers of simple units (“neurons”), each one a tiny calculation, connected to the next layer with adjustable weights.

A TINY NEURAL NETWORK
------------------------
 INPUT LAYER      HIDDEN LAYER      OUTPUT
 (raw pixels) --> (learned          --> ("this is
                   patterns)             a cat")

  o                 o
  o  ---weights--->  o   ---weights---> [cat: 0.92]
  o                 o                   [dog: 0.08]
  o                 o

🗳️ The Weighted Voting Committee Analogy: Think of a mathematical neuron as a Weighted Vote Collector. Imagine you (the output neuron) are deciding whether to go to a movie. You ask three friends (input neurons) for their opinion (Yes = 1, No = 0):

• Friend A is a film buff whose taste you trust completely (high positive weight: +0.9).
• Friend B hates everything and always complains (high negative weight: -0.8).
• Friend C is easily pleased and says yes to everything (low weight: +0.1). You multiply each opinion by its weight and sum them up. If the total passes your “bias threshold” (say, 0.5), you decide to go. In a neural network, a layer is just hundreds of these voting committees making decisions based on the inputs of the previous layer.

Here is what a standard neural network looks like when mapped visually:

graph LR
    subgraph Input Layer
        I1(Pixel 1)
        I2(Pixel 2)
        I3(Pixel 3)
    end

    subgraph Hidden Layer
        H1(Feature detector A)
        H2(Feature detector B)
    end

    subgraph Output Layer
        O1(Cat: 0.92)
        O2(Dog: 0.08)
    end

    I1 -->|Weight 1| H1
    I1 -->|Weight 2| H2
    I2 -->|Weight 3| H1
    I2 -->|Weight 4| H2
    I3 -->|Weight 5| H1
    I3 -->|Weight 6| H2

    H1 -->|Weight 7| O1
    H1 -->|Weight 8| O2
    H2 -->|Weight 9| O1
    H2 -->|Weight 10| O2

    style O1 fill:#bfb,stroke:#333

Each connection has a weight — a number representing how much one neuron’s output should influence the next. A large positive weight strengthens a signal, a small weight has less influence, and a negative weight can reduce the signal. Training the network means adjusting all of those weights, gradually, based on how wrong its guesses are — the exact same “guess, get corrected, adjust” loop from the suitcase-guessing example, just applied across millions of tiny numbers at once instead of one simple threshold.

A single layer of neurons, it turns out, can only learn fairly simple patterns. But stack several layers on top of each other — where each layer learns slightly more abstract features than the one before it — and something remarkable happens: the network starts learning genuinely complex patterns no human explicitly described.

That stacking is exactly what the next idea is named after.

Deep Learning: Going Many Layers Deep

Deep Learning is simply a neural network with many stacked layers — sometimes dozens, sometimes hundreds — instead of just one or two.

Why does depth matter so much? Because of what each layer learns to notice. In an image-recognition network, for example, the earliest layers might learn to detect simple edges and colors. The next layers combine those edges into shapes — a curve, a corner. Later layers combine shapes into parts — an eye, an ear, a wheel. The deepest layers combine parts into whole concepts — a face, a car, a cat.

WHAT EACH LAYER TENDS TO LEARN (roughly)
------------------------------------------
Layer 1:  edges, colors, simple textures
Layer 2:  corners, curves, simple shapes
Layer 3:  parts — eyes, wheels, wings
Layer 4+: whole concepts — "face," "car," "bird"

🕵️‍♂️ The Detective Agency Analogy: Why do we need layers? Imagine running a detective agency trying to solve a complex case:

• Junior Detectives (Layer 1): Scan raw evidence bags. They report only tiny details: “Found a red hair fiber,” or “There is an ink smudge here.”
• Investigators (Layer 2): Look at the junior detectives’ reports. They combine details: “A red hair plus an ink smudge indicates a suspect who signs documents with a fountain pen.”
• Inspectors (Layer 3): Take the investigators’ files: “The suspect is left-handed and works in publishing.”
• Police Chief (Output Layer): Evaluates the high-level concept: “Arrest Suspect X.” Without the hierarchy, the Chief would have to make an arrest decision looking directly at a raw pile of hair fibers and ink smudges, which is nearly impossible.

The model learns many internal features during training rather than receiving a hand-written detector for every eye, wheel, or wing. Engineers still design the architecture, prepare data, choose the training objective, and evaluate the result. The neat edge-to-shape hierarchy is a useful tendency seen in some vision networks, not a rule every layer follows perfectly.

Deep learning’s progress was helped by larger datasets, better algorithms, improved software, and powerful hardware. Some tasks use labeled datasets; language-model pretraining can create targets from the text itself. Model sizes also vary enormously: a small network may run on a phone, while a frontier model may require large accelerator clusters.

Production Insight: Why GPUs? A CPU has a smaller number of flexible workers that are good at varied, sequential tasks. A GPU has many workers designed to perform similar numerical operations in parallel.

Neural networks repeatedly perform large matrix operations containing many independent calculations. GPUs can divide that work across many processing units. The speed advantage depends on the model, batch size, software, and hardware; a GPU is not automatically faster for every program.

🧠 The Big Idea Depth is what let networks stop learning shallow surface patterns and start learning layered, hierarchical concepts — the same way you don’t understand a novel word-by-word in isolation, but build meaning up from letters, to words, to sentences, to ideas.

Checkpoint: A neural network is not a collection of hand-written rules. It is a set of connected calculations whose weights are adjusted during training. More layers give the network more opportunities to build complex patterns from simpler ones.

Teaching Machines to Handle Language

Vision and language are both hard, but language is hard in its own uniquely stubborn way. A sentence isn’t just a bag of words — meaning depends on order, context, tone, and things left unsaid. “I could care less” and “I couldn’t care less” are technically opposites but mean the same thing to most English speakers. Try writing rules for that.

Natural Language Processing (NLP) is the branch of AI aimed squarely at this problem: getting machines to work with human language — understanding it, generating it, translating it, summarizing it.

Early NLP, like early AI in general, leaned on hand-written rules and rigid statistical tricks — count how often words appear near each other, guess at grammar with fixed templates. It got you only so far, for exactly the reasons you’d now expect: language is too varied, too context-dependent, too full of exceptions for a fixed rulebook.

Deep learning changed the trajectory here too — but language had one particular challenge that image recognition didn’t: order and distance matter enormously. “The trophy didn’t fit in the suitcase because it was too big” — does “it” refer to the trophy or the suitcase? A human resolves this instantly using context from earlier in the sentence, sometimes from many words back. Early deep learning models for language struggled badly with exactly this kind of long-range context — they tended to “forget” information the further back in a sentence it appeared.

🔑 The Keyhole Reading Analogy: Imagine reading a book through a tiny keyhole that only lets you see one word at a time. When you read the word “it”, you can’t see any of the words that came before or after. You have to rely entirely on your short-term memory to remember what was in the previous room. This is how early language models (like RNNs and LSTMs) processed text: sequentially, word-by-word. By the time they reached the end of a long paragraph, they had forgotten the subject introduced at the start.

This is the bridge from learning about images to learning about language: a deep network can learn useful layers of patterns, but a language system must also decide which earlier words matter for the meaning of the current word. That problem leads to attention and Transformers.

Solving that specific problem is what set off the next big leap.

Transformers: The Main Architecture Behind Modern LLMs

In 2017, researchers introduced the Transformer, an architecture built around attention. A simple intuition is that each token can combine information from other allowed tokens and learn how strongly those tokens should influence its new representation.

“Allowed” matters. A model reading an entire input can often use information from both directions. A model generating text normally uses a causal mask, which prevents the current position from looking at future tokens that have not been generated yet.

Go back to that ambiguous sentence: “The trophy didn’t fit in the suitcase because it was too big.” An attention-based model may use information from “trophy,” “suitcase,” and other tokens to represent “it.” The exact pattern varies by model, layer, and attention head; one attention score alone is not a complete explanation of the answer.

An easy first picture is a highlighter. When the model is processing the word “it,” attention acts like a set of learned highlights over the other words. It may give a strong highlight to “trophy,” a weaker highlight to “suitcase,” and combine those signals to help decide what “it” refers to. The real mechanism uses numbers and matrix operations rather than a literal highlighter, but the purpose is similar: decide which surrounding information deserves more influence.

ATTENTION (SIMPLIFIED)
-------------------------
"The trophy didn't fit in the suitcase because it was too big"
                                              ↑
                          model learns to connect "it" strongly
                          back to "trophy" — regardless of distance

Here is how attention maps the links between words visually:

graph LR
    W1[The] --> W2[trophy]
    W2 --> W3[didnt]
    W3 --> W4[fit]
    W4 --> W5[in]
    W5 --> W6[the]
    W6 --> W7[suitcase]
    W7 --> W8[because]
    W8 --> W9[it]
    W9 -.->|"stronger link (illustration)"| W2
    W9 -.->|"weaker link (illustration)"| W7
    W9 --> W10[was]
    W10 --> W11[too]
    W11 --> W12[big]

    style W2 fill:#dfd,stroke:#333,stroke-width:1px
    style W9 fill:#fdd,stroke:#333,stroke-width:2px

This architecture — the Transformer — turned out to be extraordinarily good at two things at once: understanding long-range context, and being trained efficiently on massive amounts of text in parallel, rather than word-by-word in strict sequence. That second point mattered enormously in practice — it’s part of what made it feasible to train models on a scale nobody had attempted before.

Production Milestone: “Attention Is All You Need” In 2017, Google researchers published “Attention Is All You Need.” Compared with the recurrent sequence models common at the time, its design allowed more training work across sequence positions to run in parallel. This improved efficiency and helped make later scaling practical. Exact training times for proprietary models are not generally public, so the architectural advantage matters more than a dramatic unverified number.

Checkpoint: Attention helps a model use relevant words from the surrounding context. Transformers made that process efficient enough to train on very large collections of text.

That scale is exactly what produced the systems most people now think of when they hear “AI.”

A Short Note About Tokens

An LLM does not read a sentence as a human sees it. Before text enters the model, it is split into small pieces called tokens. A token may be a whole word, part of a word, punctuation, or a space-related piece.

"unhelpful" -> "un" + "help" + "ful"

The exact pieces depend on the tokenizer, but the important idea is simple: an LLM usually predicts the next token, not always the next complete word. This is why later tutorials talk about token counts, context windows, and tokenization.

❓ Why not just use characters or whole words?

• A whole-word vocabulary becomes very large and handles new words, names, spelling variations, and multiple languages poorly.
• Characters keep the vocabulary small but produce longer sequences, increasing the work required to connect distant pieces.
• Subword tokens are a practical compromise: a familiar word may stay whole, while an unfamiliar word can be assembled from smaller pieces.

Production Reality: The Practical Impact of Tokens

1. Usage and billing: Many model APIs measure input and output in tokens. Pricing units and rates depend on the provider and model.
2. Spelling and counting: Tokenization is one reason language models can find character-level tasks awkward, but exact tokens differ across tokenizers and it is not the only cause of every error.
3. Context budgets: A model can process only a limited amount of context in one request. An application may reject excess content, truncate it, summarize earlier material, or retrieve selected information. Even content that fits may not be used equally well.

Large Language Models: Scale Changes Everything

A Large Language Model (LLM) is a language model with many learned parameters, trained using large datasets and substantial computation. Most widely used LLMs today are Transformer-based. A common pretraining objective for generative LLMs is predict the next token, although architectures and training mixtures vary.

That’s genuinely it, at the mechanical level. Given “the cat sat on the,” predict a likely next token such as “mat.” Do that job well enough across a huge amount of text, and something surprising happens: to predict accurately, the model has to capture many patterns in the writing it sees.

For example:

• To complete “Python uses a for loop to…”, the model needs to learn common facts and coding patterns about Python.
• To complete “The glass fell because it was…”, the model needs to learn a likely relationship between an event and its cause.

This does not mean the model understands exactly like a human. It means that predicting language well forces the model to represent many regularities in language, facts, and the way people explain ideas.

💡 Think about it: Many capabilities are not installed as one hand-written rule per skill. They develop from patterns in training data and later post-training. People still shape the result through data selection, demonstrations, feedback, evaluation, and product design.

📱 The Autocomplete on Steroids Analogy: Think of an LLM as the predictive text keyboard on your phone. If you type “I am going to the…”, your phone suggests “store” or “gym” based on your past messages. Next-token prediction resembles autocomplete at a high level, but an LLM is much more complex. For example:

1. It looks at thousands of words of context instead of just the last two.
2. It uses a large neural network with many learned parameters rather than a simple lookup table.
3. Its training mixture may include selected public, licensed, human-created, code, image, audio, or other data. Providers do not always publish every detail.

This is where the earlier ideas resurface. Developers no longer need to hand-write every grammar rule or fact, but knowledge acquisition has not vanished. It has changed into choosing and governing data, objectives, feedback, evaluations, and retrieval sources. Learned parameters replace many explicit rules, while ordinary software rules still surround the model.

For the freshers: you don’t need to understand the math behind this yet — that’s what later tutorials are for. For now, hold onto the shape of the idea: a very large neural network, built from stacked attention layers, trained on an enormous amount of text, to predict what comes next.

For the experienced folks: yes, this deliberately skips deeper architectural detail — positional encoding and multi-head attention — on purpose. Those get their own dedicated deep dive later, once this map is firmly in place.

[!IMPORTANT] Production Concepts: Base vs. Instruction-Tuned Models When you download an open-source model or use an API, you must understand the distinction between:

• Base models: Pretrained primarily to continue or model data. They can be useful foundations for research and further training but may not reliably follow conversational instructions.
• Instruction-tuned or chat models: Post-trained to respond more helpfully. Supervised fine-tuning, preference optimization, reinforcement learning, synthetic data, and safety training are possible ingredients; providers use different recipes.

Chat models are usually suitable for assistants, but “always” would be too strong. A specialized application may use a base model, embedding model, classifier, or fine-tuned model instead.

Checkpoint: An LLM is a large Transformer trained to predict the next token. Its learned weights are created during training; during inference, those weights are used to produce one prediction at a time.

Generative AI: From Predicting to Creating

Once a model becomes good at predicting what comes next, it can generate new content one piece at a time by repeatedly predicting, selecting, and appending another token. The loop is simple to describe, but building and running the model still requires computation.

The generation loop looks like this:

1. Start with a prompt
2. Predict the next token
3. Add that token to the text
4. Predict the next token again
5. Repeat until the response is complete

Visualizing this generation loop:

graph LR
    Input["Prompt: 'The cat sat on the'"] --> Model["LLM Predictor"]
    Model --> NextToken["Predicted: 'mat'"]
    NextToken --> Loop["Append token to prompt"]
    Loop --> Input

That’s the essence of Generative AI — models that don’t just classify or label things, but create new text, images, audio, or code. The same underlying principle — learn the patterns in existing data, then use those patterns to produce something new — extends well beyond text. Image-generation models learn the patterns in millions of pictures; music models learn the patterns in recorded audio. Different domain, same fundamental shift: from a machine that recognizes, to a machine that produces.

Production Optimization: The Need for Streaming Because LLMs generate text token-by-token (an “autoregressive” loop), generating a 500-token response requires running the entire massive neural network 500 times in sequence. This can take several seconds. To prevent users from staring at a blank loading screen, production apps use streaming. By sending each token to the client’s screen as soon as it is predicted, the application feels fast and interactive, significantly improving user experience despite the underlying latency.

Modern AI: From Answering to Acting

The newest chapter in this story is still being written, but its shape is already clear. Instead of a model simply answering a question in one shot, modern systems increasingly:

• Break a goal into steps — plan a sequence of actions rather than produce one static answer
• Use tools — search the web, run code, query a database, call another program
• Check their own work — evaluate whether a step succeeded before moving to the next one
• Act semi-independently across multiple steps toward a broader goal

This is often called agentic AI. An agentic application usually combines a model with instructions, tools, state, permissions, and a loop that observes results before choosing another action. The model is one component, not the whole agent.

goal → model suggests action → permission check → tool runs
  ↑                                           |
  └──────── application observes result ──────┘

The permission check matters. A system that can send messages, spend money, or change data should not receive unlimited authority merely because it generates convincing text.

🤖 The Chatbot vs. Agent Analogy:

• Chatbot (Non-agentic): Imagine a researcher sitting at a desk. You ask them, “Which phone has the best camera?” They instantly reply with a summary from memory.
• Agent (Agentic): You ask the same researcher, “Compare the camera specs of the top 3 phones, find their prices at local stores, and compile a budget spreadsheet.” The researcher stands up, searches the web, writes a Python script to scrape store APIs, executes it, checks the output, fixes a code error, creates a spreadsheet, and hands you the file.

THE FULL CHAIN
-----------------
Rule-based systems      (a human writes every rule by hand)
        ↓
Machine Learning        (the machine learns rules from data)
        ↓
Neural Networks         (loosely brain-inspired learning structures)
        ↓
Deep Learning           (many stacked layers, learning abstract concepts)
        ↓
NLP + Transformers      (solving long-range context in language)
        ↓
Large Language Models   (transformers trained at massive scale)
        ↓
Generative AI           (using prediction to create new content)
        ↓
Agentic AI              (models that plan, use tools, and act)

Notice the shape of the whole story: every single arrow exists because the thing above it hit a wall. Fixed rules couldn’t scale to fuzzy, ever-changing problems, so machines learned to find patterns themselves. Shallow patterns couldn’t capture complex concepts, so networks went deeper. Deep networks still forgot long-range context, so attention fixed that. Predicting words well enough turned out to be enough to generate whole new content. And answering questions well enough turned out to be a foundation for actually acting on them.

This chain is a learning map, not a claim that each newer method replaced everything before it. Modern products frequently combine rules, search, databases, machine-learning models, and human review.

[!WARNING] Production Challenges of Agentic AI While agents are highly capable, they introduce severe production challenges:

• High Cost & Latency: An agentic workflow can take minutes to run and make dozens of LLM calls, costing significantly more than a single-turn chat response.
• Reliability & Loops: Agents can easily get stuck in infinite logic loops (e.g., correcting code, generating a new error, and trying the exact same fix again).
• Security Risk: Giving an agent tool-use capabilities (like executing shell code or updating database entries) poses critical security risks. Robust sandboxing and user-in-the-loop approvals are essential production guardrails.

What AI Does Not Guarantee

Producing a fluent answer is not the same as proving that the answer is correct.

• AI can be wrong. Generative models can produce unsupported statements, often called hallucinations.
• AI can repeat bias. Training data and product decisions can create unfair results across groups.
• AI does not prove a human-like mind. Words such as “learn,” “understand,” and “attention” describe behavior or mathematical mechanisms; they do not prove consciousness.
• AI should not receive unnecessary private data. Prompts, files, logs, and retrieved documents need suitable privacy controls.
• Human responsibility remains. High-impact medical, legal, financial, safety, or employment decisions need domain experts, evaluation, and oversight.

Use this five-question check:

1. What exact job should the AI perform?
2. What evidence shows it performs that job well?
3. What happens when it is wrong?
4. Which data and permissions does it receive?
5. Who can review, stop, or correct it?

How GPT and Gemini Fit Into This Map

GPT and Gemini are model and product families, not synonyms for all AI.

• OpenAI describes GPT-4 as a large multimodal model. Its base model was pretrained for next-token prediction and then post-trained to better follow user intent. OpenAI also documents limitations including hallucinations, bias, and adversarial prompts.
• Google’s Gemini technical report describes a family trained across text, images, audio, and video, with sizes intended for different computing needs. It also describes post-training for quality, capabilities, alignment, and safety.

large datasets + neural networks + Transformer-style processing
                          ↓
                     base model
                          ↓
        post-training + evaluation + safety work
                          ↓
      application with prompts, tools, retrieval, and guardrails

The exact data mixtures, architectures, parameter counts, and training procedures are not fully public for every proprietary model. Unofficial precise numbers should not be treated as verified facts.

Primary References

• Alan Turing: Computing Machinery and Intelligence
• Dartmouth proposal for the 1956 AI research project
• Attention Is All You Need
• OpenAI: GPT-4 research
• Google DeepMind: Gemini technical report

The Journey Is Just Beginning

We started by asking whether a machine could do more than blindly follow instructions. For most of computing history, the honest answer was no — just faster, more tireless obedience, wired together by hand.

Then, piece by piece, that changed. Machines learned to find patterns instead of following hand-written ones. Networks learned to stack those patterns into deep, layered understanding. Attention taught them to hold context across long stretches of language. And scale turned that into something that can write, reason, create, and now, increasingly, act.

So — where does this go next?

In the tutorials ahead, we’ll open up each link in this chain properly:

• How Machine Learning actually trains a model, step by step, with real numbers
• Neural Networks from the ground up — what’s really happening inside those layers
• Why Deep Learning changed everything, and what made it finally take off
• How Transformers really work — attention, tokens, and the mechanics behind most widely used LLMs
• What’s happening inside an LLM when it generates its very next word
• How Generative AI creates — text, images, and beyond
• What AI Agents actually are, and how they plan, use tools, and get things done

You’ve got the map now. Next, we go explore the terrain — one concept, one “aha,” at a time.