Seeing What Your Agent Actually Did

Recall Module 17’s astream_events — a real, useful way to watch an agent’s loop live, in your terminal, while it’s actually running. That’s genuinely valuable for development, and genuinely insufficient for anything beyond it: it disappears the moment your program finishes, gives you no history to look back on, and offers nothing for comparing behavior across many real requests over time.

Why print(response) breaks down at real scale

Recall the full request-execute-respond cycle from Module 13, and the multi-round loop from Module 14. A single real request to a deployed agent might involve several model calls, several tool executions, and real, meaningful decisions at each step. When a user reports “the agent gave me a wrong answer,” print(response) shows you only the final output — none of the actual reasoning, tool calls, or intermediate results that produced it. Recall Module 17’s own common mistake: debugging from the final answer alone throws away exactly the information you need.

Enter LangSmith, briefly and practically

Recall Module 2’s honest framing: LangSmith is a separate, companion tool for watching and recording what your LangChain application actually did. This module gives it just enough treatment to support your real, everyday development — full production observability deserves the dedicated treatment your later Production AI Engineering course will give it.

Example 1: enabling tracing with almost no code

import os

os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_API_KEY"] = "your-langsmith-api-key"

from langchain.chat_models import init_chat_model
from langchain.tools import tool
from langchain.agents import create_agent

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"It's 22°C and sunny in {city}."

agent = create_agent(model=init_chat_model("openai:gpt-4o-mini"), tools=[get_weather])
result = agent.invoke({"messages": [{"role": "user", "content": "What's the weather in Cairo?"}]})
print(result["messages"][-1].content)

Notice the actual agent code is completely unchanged from every previous module — tracing activates through environment variables alone, echoing Module 3’s .env pattern. Every model call, every tool execution, and every intermediate step from this run is now recorded and viewable in LangSmith’s own web interface, as a real, persistent trace.

Example 2: what a trace actually shows you

A real LangSmith trace for the run above displays, concretely:

• The exact input sent to the model, and its exact reply, including the tool call it requested.
• The exact arguments get_weather was called with, and its exact returned result.
• The real, measured latency — recall the Infrastructure phase’s discussion of TTFT and total latency — of each individual step.
• The real, exact token counts and cost for each model call, recall Module 18’s honest cost-tracking concern.

This is genuinely the same information astream_events showed you live, in Module 17 — now persisted, searchable, and comparable across every real request your application has ever handled, not just the one currently running in front of you.

Example 3: adding your own custom metadata to a trace

from langchain.chat_models import init_chat_model
from langchain.agents import create_agent

agent = create_agent(model=init_chat_model("openai:gpt-4o-mini"), tools=[])

result = agent.invoke(
    {"messages": [{"role": "user", "content": "Summarize our return policy."}]},
    config={"metadata": {"user_id": "u_42", "feature": "support_chat"}},
)

Real metadata like user_id and feature makes a trace genuinely searchable later — “show me every trace from this specific user” or “show me every request from this specific feature” becomes a real, practical query, rather than manually scanning through undifferentiated logs.

Why this matters directly for debugging a real agent

Recall Module 17’s GraphRecursionError and Module 16’s business-tool-suite agent. When either of these misbehaves in production — hits its recursion limit unexpectedly, or calls tools in an unexpected order — a real trace shows you exactly which step went wrong, with exactly what data, rather than leaving you to guess or attempt to painstakingly reproduce the failure locally.

Common mistakes worth avoiding

Only enabling tracing after a problem has already happened. Tracing is genuinely cheap to leave on continuously in a real, deployed application — the value comes specifically from having a real, historical record available before you know you’ll need it, not scrambling to enable it after a user has already reported an issue.

Never adding meaningful metadata, then struggling to find relevant traces later. Recall Example 3 — a production system handling thousands of real requests, with no metadata attached, makes finding “the specific trace for this specific user’s specific complaint” genuinely difficult, even though the data technically exists somewhere.

What you should take away from this module

• print(response) shows only a final answer; a real trace shows every step that produced it — every model call, every tool execution, every intermediate result.
• Enabling LangSmith tracing requires no changes to your actual agent code — just environment variables, following the same pattern as Module 3’s API key setup.
• Custom metadata, attached via config, makes traces genuinely searchable by real, meaningful criteria later.
• This is genuinely enough LangSmith to support real, everyday development — full production monitoring gets its own, deeper treatment later in your broader curriculum.

Where this goes next

The next module covers Testing — writing real, runnable tests for prompts, structured output, tools, chains, and agents, so you can verify your application actually behaves correctly before a real user ever encounters a problem a trace would only reveal afterward.