Section 9 — Tool Calling

Begin with the problem

From talking to doing

A model can suggest an action, but your Java application must decide whether to execute it. Tool calling connects a structured model request to controlled application code.

model requests tool → application validates and runs it → result returns to model

What you will learn

• Define tools and their input schemas.
• Follow the tool-calling loop.
• Understand that the application—not the model—executes code.
• Add authorization, validation, limits, and approvals.

Current official reference: Spring AI documentation for this topic. The examples below primarily preserve the stated 1.1.x course target. Where Spring AI 2.0 differs, the text must treat that behavior as version-specific rather than universal.

(Continues from Section 8. Target: Spring AI 1.1.x / Spring Boot 3.5.x.)

9.1 Why Tool Calling Is “Just Reflection + JSON Schema Generation”

Current 2.0 behavior to know first

Spring AI 2.0 makes tool calling part of the ChatClient advisor chain. The auto-registered ToolCallingAdvisor sends tool definitions, receives a model’s tool request, asks ToolCallingManager to execute the matching callback, appends the result, and repeats until the model returns a normal answer.

ChatClient
   ↓
ToolCallingAdvisor → model asks for a tool
   ↓
ToolCallingManager → your application executes approved Java code
   ↓
tool result returns to the model → final answer

This matters for safety: the model proposes a name and arguments, but your application still owns execution, authorization, validation, timeouts, and approval gates.

Beginner primer: if tool/function calling is new to you, read the glossary entry first. Short version: an LLM can’t execute code or query a live system directly — it can only generate text. Tool calling lets you describe available Java methods to the model; if the model decides a method would help, it replies with a structured request to call it with specific arguments, your code actually runs the method, and the result is fed back so the model can use it in its final answer.

Spring AI’s specific contribution: turning a plain Java method (or @Bean of type Function/BiFunction) into an LLM-invocable tool via annotation-driven reflection and automatic JSON Schema generation — no manual schema authoring, no manual argument-parsing boilerplate.

Real-world analogy — Company Directory + Extension Routing: A tool is like a company phone directory entry: name, description of what that person/department handles, and what information you need to provide when you call them (their “extension” is the JSON Schema for their expected input). The LLM is the caller who reads the directory (tool descriptions), decides who to call based on the request, dials with the right info (JSON arguments), and gets a response back — Spring AI is the directory-and-switchboard system generating that directory automatically from your actual Java methods, and physically routing the call via reflection.

Analogy: The Company Directory & Switchboard Extension Routing Think of enabling tool calling inside your Spring Boot codebase as setting up an automated office switchboard:

• The Directory (Schema Gen): Spring AI looks at your @Tool annotated Java methods and publishes an official corporate directory (JSON Schema). This directory lists: “Department: Weather; Call handler: getCurrentWeather; Info required: city (string), unit (string).”
• The Customer (The LLM): The customer speaks to the operator: “What’s the temperature in Bengaluru?” The operator reads the directory list and decides to dial extension getCurrentWeather with payload {"city": "Bengaluru"}.
• The Switchboard (ToolCallingManager): Spring AI intercepts the call, uses reflection to route to your local Java class method, executes the database/API logic, and routes the department’s response sheet (WeatherResult) back to the customer’s ears.

📊 Visual Flowchart: The Reflection and Tool Execution Lifecycle Loop

Here is how text prompts trigger model-driven tool calls, running local Java code via reflection:

sequenceDiagram
    autonumber
    actor User as ChatClient Caller
    participant Spring as Spring AI Framework
    participant Model as LLM ChatModel
    participant Tool as Java @Tool Bean

    User->>Spring: chatClient.prompt().user("Weather in Bengaluru")
    Note over Spring: Reflects on @Tool method signatures<br>and builds JSON Schema parameters
    Spring->>Model: Sends prompt + tool definitions list
    Note over Model: Decides a tool is needed.<br>Responds with a structured request.
    Model->>Spring: Return AssistantMessage (tool_calls: name, args)

    rect rgb(240, 248, 255)
        Note over Spring: ToolCallingManager intercepts.<br>Deserializes args & runs Java method.
        Spring->>Tool: Invoke weatherTools.getCurrentWeather("Bengaluru") via reflection
        Tool->>Spring: Return WeatherResult object
    end

    Spring->>Model: Sends ToolResponseMessage (serialized result)
    Note over Model: Processes the tool's output.<br>Generates final answer.
    Model->>Spring: Return final response text
    Spring->>User: "The weather in Bengaluru is 24°C"

---

9.2 The @Tool Annotation Path

@Component
public class WeatherTools {

    private final WeatherApiClient weatherApiClient;

    public WeatherTools(WeatherApiClient weatherApiClient) {
        this.weatherApiClient = weatherApiClient;
    }

    @Tool(description = "Get the current weather conditions for a specific city")
    public WeatherResult getCurrentWeather(
            @ToolParam(description = "City name, e.g. 'Bengaluru' or 'San Francisco'") String city,
            @ToolParam(description = "Temperature unit: 'celsius' or 'fahrenheit'", required = false)
            String unit) {
        return weatherApiClient.fetch(city, unit == null ? "celsius" : unit);
    }
}

chatClient.prompt()
        .user("What's the weather like in Bengaluru right now?")
        .tools(new WeatherTools(weatherApiClient))   // or auto-discovered, see §9.3
        .call()
        .content();

9.2.1 What Happens Internally

.tools(weatherToolsInstance)
        │
        ▼
1. ToolCallbackResolver reflects over the instance's methods,
   finds @Tool-annotated methods
        │
        ▼
2. For each: JSON Schema generated from method signature —
   parameter types + @ToolParam descriptions → OpenAPI-style schema.
   Records/POJOs as parameters get recursively schema'd (nested
   object support); primitives/String/enum map to schema primitives/enums
        │
        ▼
3. ToolCallback wraps: {name, description, inputSchema, invoke(Map args)}
   registered into the request's ChatOptions.toolCallbacks
        │
        ▼
4. Model receives tool definitions in its request (each provider's
   own tool-schema wire format — Section 4's translation-layer point
   applies here directly)
        │
        ▼
5. Model responds with tool_calls in AssistantMessage (name + JSON args)
        │
        ▼
6. ToolCallingManager.executeToolCalls():
   - resolves ToolCallback by name
   - deserializes JSON args → method parameter types (Jackson)
   - invokes the actual Java method via reflection
   - serializes return value → ToolResponseMessage
        │
        ▼
7. Loop back to model call with tool result appended (internal
   tool-execution loop from Section 1/3), UNLESS
   internalToolExecutionEnabled(false)

---

9.3 Bean-Based Discovery vs. Explicit Registration

Two registration styles:

// Style 1 — explicit, per-request tool objects (shown above)
.tools(new WeatherTools(weatherApiClient))

// Style 2 — auto-discovered Spring beans, registered globally via
// ToolCallbackProvider, resolved by name at call time
@Bean
public ToolCallbackProvider weatherToolCallbackProvider(WeatherTools weatherTools) {
    return MethodToolCallbackProvider.builder()
            .toolObjects(weatherTools)
            .build();
}

// then reference by name string instead of object instance:
.toolNames("getCurrentWeather")

Production guidance: explicit per-request .tools(...) registration is generally preferable for anything with side effects or tenant-scoped behavior — it keeps the set of available tools for a given call visible and auditable at the call site, rather than relying on globally-registered beans whose availability might silently change as the application evolves. Global bean-based discovery is more convenient for a small, stable set of pure read-only utility tools shared broadly.

---

9.4 Function-Based Tools (Non-Annotation Style)

For tools defined outside a component you control (or dynamically constructed):

@Bean
public FunctionToolCallback<OrderLookupRequest, OrderLookupResult> orderLookupTool(
        OrderService orderService) {
    return FunctionToolCallback.builder(
                    "lookupOrder",
                    (OrderLookupRequest request) -> orderService.findOrder(request.orderId()))
            .description("Look up an order by its order ID")
            .inputType(OrderLookupRequest.class)   // record — schema generated from it
            .build();
}

public record OrderLookupRequest(String orderId) {}

This is functionally equivalent to @Tool but useful when the tool logic is a lambda/method reference rather than an instance method on a Spring-managed bean, or when you’re constructing tools dynamically (e.g., building a tool per tenant configuration at runtime).

---

9.5 JSON Schema Generation — Detail Worth Knowing

Spring AI generates schemas from Java types using its own lightweight reflection-based generator (not a full JSON Schema library) — this covers primitives, String, enums, records, POJOs with getters, List<T>, and nested objects, but has real limits:

• Generic types (List<Map<String, CustomType>>) may not schema cleanly — prefer flat, well-typed records for tool parameters.
• Polymorphic types (an interface parameter with multiple implementations) aren’t automatically disambiguated — the model has no way to know which concrete shape to produce; avoid interface-typed tool parameters.
• required defaults to true for a @ToolParam unless explicitly marked required = false — omitting this on really optional parameters produces a schema that pressures the model to always supply a value, sometimes fabricating one.

// GOOD — flat record, clear required/optional split
public record RefundRequest(
        String orderId,
        @Nullable String reason // combine with @ToolParam(required = false)
) {}

// RISKY — nested generic collection, unclear schema, avoid for tool params
public record BadRequest(Map<String, List<Object>> data) {}

---

9.6 Error Handling

@Tool(description = "Process a refund for an order")
public RefundResult processRefund(@ToolParam(description = "Order ID") String orderId) {
    try {
        return refundService.process(orderId);
    } catch (OrderNotFoundException e) {
        // returning a structured error result the MODEL can read and
        // react to (e.g., ask the user to double check the order ID)
        // is usually better than letting the exception propagate raw
        return RefundResult.failed("Order not found: " + orderId);
    } catch (RefundAlreadyProcessedException e) {
        return RefundResult.failed("This order was already refunded.");
    }
}

Key design decision: should a tool failure be (a) an exception that aborts the whole ChatClient call with a 500, or (b) a structured error result fed back to the model so it can recover conversationally (“that order ID doesn’t exist, can you double check it?”)? For user-facing conversational tools, (b) is almost always the better UX — the model can gracefully handle a “not found” and ask a clarifying question, whereas an uncaught exception surfaces as a broken response to the end user. Reserve uncaught exceptions for really unrecoverable infrastructure failures (database down), not expected business-logic outcomes (order not found, validation failure).

Spring AI also supports a ToolExecutionExceptionProcessor SPI to centralize this translation (exception → model-readable error message) instead of try/catch in every tool method:

@Bean
public ToolExecutionExceptionProcessor toolExceptionProcessor() {
    return exception -> {
        if (exception.getCause() instanceof OrderNotFoundException) {
            return "The specified order was not found. Ask the user to verify the order ID.";
        }
        return "An unexpected error occurred while executing the tool.";
    };
}

---

9.7 Retry for Tool Execution

Tool-level retry is distinct from model-call retry (Section 3) — a tool might call a flaky downstream service:

@Tool(description = "Fetch current inventory count for a SKU")
@Retryable(
        retryFor = TransientDataAccessException.class,
        maxAttempts = 3,
        backoff = @Backoff(delay = 500, multiplier = 2)
)
public InventoryResult checkInventory(@ToolParam(description = "Product SKU") String sku) {
    return inventoryService.getCount(sku);
}

Standard Spring Retry (@Retryable) applies directly here since a @Tool-annotated method is, structurally, just a Spring-managed bean method — no Spring-AI-specific retry mechanism is needed at the tool level, reinforcing that tool calling is deliberately built on ordinary Spring idioms rather than a parallel framework.

---

9.8 Security — Tool Execution Is Code Execution Triggered by an LLM

This deserves explicit emphasis: a tool call is the model deciding to execute your code with model-chosen arguments. Treat every tool the same way you’d treat any externally-triggered code path — because from a trust-boundary perspective, that’s exactly what it is; the “external caller” happens to be a language model interpreting user input, which itself may contain injected instructions (prompt injection, covered fully in Section 15).

@Tool(description = "Send an email on behalf of the current user")
public void sendEmail(
        @ToolParam(description = "Recipient email") String to,
        @ToolParam(description = "Subject") String subject,
        @ToolParam(description = "Body") String body) {

    // MANDATORY: authorization check inside the tool method itself —
    // never trust that "the model wouldn't call this inappropriately"
    if (!currentUserCanSend(to)) {
        throw new AccessDeniedException("Not authorized to email this recipient.");
    }

    // MANDATORY: validate/sanitize model-supplied content before
    // it reaches a real side effect
    emailService.send(currentUserEmail(), to, sanitize(subject), sanitize(body));
}

For any tool with side effects (sends, writes, payments, deletions), the production-grade pattern is:

1. internalToolExecutionEnabled(false) so execution isn’t fully automatic and unattended.
2. A human-approval gate before the tool actually runs, for sufficiently sensitive actions (Section 17 covers human-in-the-loop patterns in the multi-agent context).
3. Authorization checks inside the tool method itself, never assumed from prompt-level instructions (“only email people in the same company” as a system-prompt instruction is not a security control — a determined prompt injection can override it; the actual authorization check must be code, not prompt text).

---

9.9 Common Mistakes

1. Treating system-prompt instructions as security controls — “don’t call this tool unless X” in a system prompt is guidance, not enforcement; enforce in code.
2. Letting tool exceptions propagate uncaught for expected business-logic failures, breaking the conversational flow instead of letting the model recover gracefully.
3. Nested generic or polymorphic tool parameters producing ambiguous schemas the model can’t reliably fill correctly.
4. Forgetting required = false on really optional parameters, pressuring the model to fabricate values.
5. No authorization check inside side-effecting tools, relying entirely on the model’s “good judgment” about when to call them.
6. Leaving internalToolExecutionEnabled(true) (the default) for high-risk tools without a human-approval gate.

---

9.10 Debugging

logging:
  level:
    org.springframework.ai.tool: DEBUG

This surfaces the generated JSON Schema for each registered tool — the fastest way to diagnose “the model keeps calling my tool with wrong/missing arguments” is to look at the actual schema it was given, not just the Java method signature you wrote; a mismatch between intent and generated schema (e.g., a field you assumed was clearly named turning out ambiguous without a description) is a very common root cause.

---

9.11 Interview Questions

1. How does Spring AI generate a JSON Schema from a @Tool-annotated method’s signature, and what are its real limitations (generics, polymorphism)?
2. Why is internalToolExecutionEnabled a meaningful safety control, and when should it be disabled?
3. What’s the architectural difference between explicit per-request .tools(...) registration and global bean-based ToolCallbackProvider discovery, and which is preferable for side-effecting tools?
4. Why is a system-prompt instruction like “only call this tool if authorized” not a real security control?
5. Design an authorization strategy for a tool that sends emails on a user’s behalf, addressing both the model’s arguments and the actual authorized recipient list.
6. What’s the trade-off between letting a tool exception propagate uncaught versus returning a structured error result the model can read?
7. How does ToolExecutionExceptionProcessor centralize exception-to-model-message translation, and why is that preferable to try/catch scattered across every tool method?
8. Why does standard Spring @Retryable work directly on @Tool-annotated methods without any Spring-AI-specific retry mechanism?
9. What happens internally between a model returning tool_calls in its response and your Java method actually executing?
10. Why should required = false be set explicitly for optional tool parameters, and what happens if it’s omitted?
11. Describe a production pattern for human-in-the-loop approval before a high-risk tool call executes.
12. What’s the risk of a nested generic type like Map<String, List<Object>> as a tool parameter?
13. How would you debug a case where the model consistently calls a tool with malformed arguments?
14. What’s the difference between FunctionToolCallback and @Tool-annotated methods, and when would you prefer one over the other?
15. Why is tool calling described as “the model executing your code with model-chosen arguments,” and what trust-boundary implications does that framing carry?
16. How does prompt injection (Section 15 preview) relate to tool-calling security specifically?
17. What’s the correct place to put an authorization check for a side-effecting tool — the system prompt, the tool method, or both, and why?
18. How would you structure a tool’s return type to give the model enough information to recover gracefully from a “not found” business error?
19. What role does ToolCallingManager play in the internal tool-execution loop, and how does it relate to the loop first introduced in Section 1’s execution flow diagram?
20. Why might explicit .tools(...) registration be preferable to global bean discovery specifically for auditability?

---

9.12 Best Practices Checklist

• Never rely on prompt-level instructions as the sole authorization control for a side-effecting tool — enforce in code.
• Set internalToolExecutionEnabled(false) with a human-approval gate for high-risk tools (sends, writes, payments, deletions).
• Return structured error results for expected business-logic failures; reserve uncaught exceptions for genuine infrastructure failures.
• Explicitly mark optional @ToolParams with required = false.
• Keep tool parameter types flat (records/POJOs), avoiding nested generics and polymorphic interface types.
• Prefer explicit per-request tool registration for auditability on anything with side effects.
• Apply standard Spring Retry to tool methods calling flaky downstream dependencies.

---

9.13 Key Takeaways

• Tool calling is annotation-driven reflection + automatic JSON Schema generation — no separate framework, built entirely on ordinary Spring idioms.
• A tool call is code execution triggered by a model’s interpretation of (possibly adversarial) user input — treat it as an external trust boundary, not an internal function call.
• Structured error results generally beat uncaught exceptions for expected business-logic failures in conversational tools.
• internalToolExecutionEnabled is your primary lever for inserting human oversight into otherwise-automatic tool execution.
• JSON Schema generation has real limits around generics and polymorphism — design tool parameter types deliberately, not incidentally.

---

End of Section 9. Next: Section 10 — MCP Integration (Servers, Clients, Transport, Lifecycle, Tool Discovery, Authentication, Enterprise MCP).