What Is Agentic AI?

Imagine asking an AI, “Please plan my school trip.” A chatbot may give you a list of ideas. An agent can go further: it can check dates, compare places, ask for missing information, use approved tools, and pause for your permission before booking anything.

That difference—moving from answering to working toward a goal—is the heart of agentic AI.

Question → AI writes an answer

Goal → AI chooses a step → uses a tool → checks the result → chooses again

What You Will Learn

• What agentic AI means and how it differs from a chatbot or a single LLM response.
• Why an agent needs a goal, tools, observations, decisions, and a stopping condition.
• How the agent loop turns one large task into a series of smaller actions.
• Where humans, permissions, safety checks, and ordinary software fit into the system.
• When an agent is useful and when a simpler workflow is safer and cheaper.

A Current System You Can Compare With This Diagram

Google’s Gemini API now documents managed agents that can reason, execute code, manage files, and browse the web inside a sandbox. The product is more advanced than our small diagram, but the central loop is the same: receive a goal, choose an action, observe the result, and continue until a stopping condition is reached. (Google Gemini Agents overview)

The important lesson is not that every agent must use Gemini. It is that real agent products need a model plus an execution environment, tools, state, and controls; the language model by itself is only one component.

Let’s start with a situation you’ve probably already lived through, even if you haven’t called it by this name yet.

You built an LLM application for customer support. It reads a customer’s message, understands the intent, and generates a helpful, well-written response. It works. Customers are happier. The support team ships it, everyone’s pleased, and then a few weeks later, someone from the business side asks a completely reasonable question:

“Can it fix the customer’s problem, not just talk about it?”

This sounds like a small ask. It isn’t. Here’s what “fix the problem” means for a single support ticket like “My payment failed twice this week and I still haven’t been refunded”:

1. Identify the customer
2. Look up their account information
3. Check their payment and order history
4. Check the company’s refund policy for this situation
5. Decide whether a refund is justified
6. Call the refund API to issue it
7. Update the CRM so this is recorded
8. Notify the customer that it’s done
9. If anything about the situation looks unusual, escalate to a human instead of guessing

Now ask yourself honestly: is the system you already built — the one that reads a message and writes a good reply — still “just a chatbot” if it has to do all of that? And more importantly: can it do all of that, the way it’s currently built?

It can’t. Not because the underlying language model isn’t smart enough, but because of something more structural. The chatbot you built has one job: take an input, produce an output. It was never designed to look anything up, take a real action, or decide what to do next based on what it discovers along the way. Getting from “answers questions well” to “resolves the actual problem” isn’t a matter of writing a longer, cleverer prompt. It requires a fundamentally different kind of system.

That system is what this entire course is about, and this module’s only job is to give you the correct mental model for it before we go any deeper.

Why a single generation isn’t enough

To see exactly where the gap is, it helps to be precise about what an LLM and an “LLM application” are, because people use these terms loosely and the looseness hides the real problem.

An LLM — the model itself, GPT-4-class, Claude, Gemini, whatever you’re using — is fundamentally a function that takes text in and produces text out. Give it a prompt, it generates a completion. That’s the entire capability, at the most basic level. It has no memory of anything outside that one call unless you explicitly put it in the prompt, and it has no way to reach outside itself and touch anything in the real world. It can’t check a database.

It can’t call an API. It can only generate text based on the patterns it learned during training and whatever you handed it in the prompt.

An LLM application is what you build around that raw capability to make it useful: a system prompt that sets the tone and boundaries, some logic that formats the user’s message, maybe a retrieval step that pulls in relevant company documents (this is RAG, which you’ve already studied), and then a call to the model that produces a response. The shape of this, at its simplest, looks like:

Input
 ↓
LLM
 ↓
Response

This is powerful for an enormous range of tasks — answering questions, summarizing documents, drafting emails, classifying sentiment, writing code snippets. If the task is “take some input and produce a well-reasoned output,” a plain LLM application is often exactly the right tool, and building anything more complicated would be a mistake. Hold onto that, because it’s going to matter a lot more once we get further into this course.

But notice what this shape cannot do. It cannot look up the customer’s actual account, because “looking something up” means reaching outside the model into a real system, and this shape has no mechanism for that at all. It generates one response and stops. There’s no way for it to say “wait, before I answer, I need to go check something” and then go check it. The entire interaction is one shot: input goes in, output comes out, done.

This is the actual wall you hit with the refund scenario. Steps 2 through 8 all require reaching into real systems — a customer database, a payment API, a CRM — and using what comes back to inform what happens next. A single generation from a single prompt structurally cannot do this, no matter how good the underlying model is or how carefully you word the prompt.

You could try writing an enormous prompt that says “here’s the customer’s data, here’s their payment history, here’s the policy, now decide and respond” — but that requires you, the engineer, to have already gathered all of that information and jammed it into the prompt before the model even starts. The model isn’t deciding what to look up. You are.

And that defeats the entire point, because the situations that need this kind of system are exactly the ones where you don’t know in advance what needs to be checked, because it depends on what’s discovered along the way.

The missing pieces: action and decision

Two things are missing from a plain LLM application, and once you see them named, the rest of this module — and honestly, the rest of this whole course — starts to make a lot more sense.

The first missing piece is action. The model needs a way to do something in the world, not just describe what it thinks should happen. This is what a tool provides — a defined, callable capability like “look up a customer’s account” or “check payment history” that the model can invoke and get a real result back from. We’ll go deep on tools in Module 5; for now, just hold the idea that a tool is what turns “the model talking about doing something” into “the model doing it.”

The second missing piece is decision — specifically, an ongoing decision, not a one-time one. Even if you gave a plain LLM application a single tool, it would still only get to use it once, because the shape of the system stops after one response. What the refund scenario needs is a system that can call a tool, look at what came back, and then decide — decide, based on that specific result — what to do next.

Maybe it needs another tool call. Maybe it has enough information already. Maybe what it found was unexpected and it needs to change its approach entirely. That ongoing, repeated decision-making, informed by real results as they come in, is the actual core of what makes something an agent.

Put action and repeated decision-making together, wrap them in a loop that keeps going until the goal is satisfied, and you get this:

Goal
 ↓
Observe
 ↓
Reason
 ↓
Decide
 ↓
Act
 ↓
Observe result
 ↓
Continue / Stop

Compare that directly against the plain LLM application shape from earlier:

LLM Application

Input
 ↓
LLM
 ↓
Response

The LLM application shape has a beginning and an end, and nothing happens in between except one generation. The agent shape has a loop — notice how “observe result” feeds back around, rather than the diagram just ending. That loop is the entire mechanism. Everything else you’ll learn in this course — tools, planning, memory, guardrails, evaluation — exists either to make that loop possible or to make it safe and reliable.

An agent is not simply an LLM with a fancy prompt

This is worth stating directly, because it’s the single most common misunderstanding people carry into this topic, and it’s worth correcting early before it quietly shapes how you think about everything that follows.

You cannot turn a plain LLM application into an agent purely by writing a more elaborate system prompt — something like “you are an autonomous agent, think step by step, decide what to do, take action when needed.” Wording alone doesn’t create the loop. Wording alone doesn’t give the model tools to call. Wording alone doesn’t give it a way to observe a real result and use it to inform the next decision.

Without the actual mechanism — tools it can invoke, a system that executes them and returns real results, and a loop that keeps running until the goal is met — a “be an agent” prompt just produces a model that talks about acting agentically, in the same single-shot way it always did. It might even generate very convincing text describing steps it would take. It still won’t have taken any of them.

This distinction matters because it’s exactly what separates a agentic system from something that just sounds agentic. When you hear “we built an AI agent,” the honest engineering question is: does this system have tools it can call, does it observe real results, and does it make a new decision based on each result — or is it a well-prompted LLM application producing agent-flavored language with no real loop underneath it? Both exist in the wild. Only one of them can do what the refund scenario requires.

Not every AI application should become an agent

Here’s the other correction worth making early, because the instinct once you learn about agents is to reach for one everywhere, and that instinct is usually wrong.

Go back to the plain LLM application shape — input, LLM, response. A huge number of valuable AI systems are exactly this shape and should stay that way. If a customer asks “what’s your return policy?” and the honest, complete answer lives in a document you can retrieve and hand to the model, you don’t need a loop, you don’t need tools that take real-world action, and you don’t need the model deciding anything beyond how to phrase a grounded answer.

That’s a RAG application — retrieval plus generation — and it’s a better, simpler, cheaper, more predictable fit for that task than an agent would be.

The refund scenario is different specifically because it has a property the return-policy question doesn’t: the correct sequence of steps depends on information you don’t have yet, and won’t have until you go look. You don’t know in advance whether the payment failed or successfully went through twice. You don’t know whether the account is in good standing or flagged for something else entirely.

Whether a refund is even the right action depends on what gets discovered along the way — which means the sequence of steps can’t be fully written out in advance. That’s the actual signal that a task might need an agent: not “this involves an LLM,” not “this sounds complicated,” but specifically “the right next step depends on information I don’t have until I’ve taken the previous step.”

We’re going to come back to this distinction constantly throughout this course, because getting it right is one of the highest-value engineering judgments you’ll make when building these systems. Building an unnecessary agent for a task a plain LLM application or a fixed workflow could handle isn’t a neutral choice — it’s slower, more expensive, harder to test, and less predictable, for zero actual benefit. We’ll build a full decision framework for this in the very last module of this section.

For now, just notice that the question “should this be an agent?” is a real, separate engineering decision — not something you default into because agents are the more sophisticated- sounding option.

Where this term came from

It’s worth knowing, briefly, that “AI agent” isn’t a term a marketing department invented recently to make chatbots sound more impressive. The pattern — an LLM given tools and a loop to decide its own next action — became a distinct, widely-discussed engineering pattern starting around 2023, when open-source experiments like AutoGPT and BabyAGI demonstrated, publicly and somewhat chaotically, what happens when you let a model call itself repeatedly with access to tools and a goal instead of a single prompt.

Those early projects were rough — prone to looping, burning through tokens, and taking strange detours, which is exactly the kind of failure mode you’ll learn to guard against later in this course — but they made the underlying idea concrete and visible enough that the major model providers took notice.

Shortly after, OpenAI, Anthropic, and Google all built native tool- calling support directly into their model APIs — a structured, reliable way for a model to request “call this specific function with these specific arguments” instead of engineers having to parse loosely formatted text out of a free-form response and hope it matched what they expected.

That shift — from informally coaxing structured actions out of plain text to having a, purpose-built mechanism for it — is what took “agent” from an interesting experiment to something you can build reliable, production-grade systems on top of. We’ll get into exactly how that mechanism works, concretely, in Module 5.

The vocabulary you now have

Let’s name everything we’ve built up in this module, because you’ll see these words used precisely — and sometimes imprecisely — for the rest of this course.

A goal is the outcome the system is trying to achieve — not “answer this question,” but “get this customer’s issue resolved.” An action is a real thing the system does in the world, as opposed to a description of a thing. A decision is the system choosing what to do next, based on its current, clear understanding of the situation.

State is that evolving understanding — what the system has learned and done so far in this particular task, which we’ll go much deeper on in Module 7. Autonomy is how much of this decision-making the system is allowed to do without a human checking in first — a spectrum, not a switch, which gets its own full module later (Module 9).

And an agent is what you get when a model, a set of real tools, and a loop that keeps making decisions based on real results are combined to pursue a goal, rather than answer a single question.

One more term worth planting now, because you’ll meet it properly in Module 8: an agentic workflow. Not every system with some dynamic decision-making needs to be a fully autonomous agent making every choice on its own. Sometimes the right design is a mostly fixed sequence of steps with one or two points where an LLM makes a human judgment call — routing a support ticket to the right department, for instance.

That’s meaningfully different from full autonomy, and the difference matters enormously for reliability, cost, and how much you can trust the system. Keep that term in the back of your mind; we’ll build it out fully soon.

When to Use an Agent—and When Not To

Use a bounded agent when the goal is clear but the next step depends on information discovered during the task, and when approved tools can provide trustworthy feedback.

Do not use an agent when ordinary code already knows every step, one model response is enough, or a wrong action would be unacceptable without deterministic control and human approval.

Common Misconception

Incorrect idea: Agentic AI is simply a more intelligent chatbot.

Why it is incorrect: Intelligence and agency are different. Agency comes from a loop, tools, state, feedback, permissions, and stopping rules surrounding the model.

Key Takeaways

• A plain LLM application follows the shape Input → LLM → Response — one generation, no real-world action, no ongoing decision-making. This is the right architecture for a large number of tasks.
• An agent adds two things a plain LLM application structurally lacks: the ability to take real action via tools, and the ability to make a new decision based on the real result of the previous action, repeated in a loop until the goal is satisfied.
• The loop — Goal → Observe → Reason → Decide → Act → Observe result → Continue / Stop — is the actual mechanism of agentic AI. Everything else in this course builds on making that loop possible, safe, and reliable.
• An agent is not created by prompt wording alone. “Act like an autonomous agent” in a system prompt, with no real tools and no real loop underneath it, produces text that describes acting agentically — not a system that does.
• The real signal that a task needs an agent isn’t complexity or importance — it’s whether the correct next step depends on information you don’t have until a previous step has already been taken.
• Not every AI system should become an agent. Reaching for one by default, when a plain LLM application or a fixed workflow would reliably solve the task, is a real engineering cost with no corresponding benefit.
• The term “AI agent” traces to real, public engineering experiments (AutoGPT, BabyAGI, 2023) that demonstrated the pattern before major model providers built native, structured tool-calling support into their APIs to make it reliable enough for production use.

Think Like an AI Engineer

• You have a system that answers “what’s our shipping policy?” by retrieving the relevant document and generating a grounded answer. Does this need to become an agent? What would change if you turned it into one, and would that change be worth it?

• A colleague says “I made our support bot into an agent, I just told it in the prompt to act autonomously and take whatever steps are needed.” Based on what you now know, what questions would you ask them to find out whether they’ve built an agent?

• Think of a real, multi-step task from your own work — something with at least three meaningful decision points. Which steps in that task depend on information you wouldn’t have until a previous step finished? That dependency is exactly the signal this module described. Is it present in your example, or could every step be fixed in advance?

• The refund scenario at the start of this module has nine steps. Which of those nine do you think could reasonably be a fixed, predetermined step regardless of what’s discovered — and which ones require a decision made in the moment? Try to justify your answer for each one, not just guess.

Module 2 picks up exactly where this one leaves off: why did the industry need to build this pattern at all? We’ll trace the real evolution — from traditional rule-based software, through machine learning and deep learning, to LLMs, to tool calling, and finally to agents — and look closely at the specific limitation that forced each step, and the specific new problem each step introduced in exchange.