Tool Calling & Structured Output

Begin with the problem

Software cannot safely depend on a paragraph that merely looks like JSON. Structured output and tool calling create a typed boundary, but the application must still validate everything before use.

schema → model-generated structure → parse → validate → execute/use → handle error

What you will learn

Distinguish structured final output from a tool-call request.
Validate schemas, arguments, tool results, and business rules.
Recover safely from malformed or incomplete model output.

Current production grounding: Google’s Gemini tools documentation distinguishes managed built-in tools from custom functions executed by the application.

These links document current public features and practices. They do not reveal a provider’s private implementation or guarantee that every product uses identical defaults.

Topic-specific reference: OpenAI’s Structured Outputs API reference documents JSON Schema response formats and strict schema adherence.

1. The Engineering Problem

Your downstream code needs a specific data shape — a valid JSON object with specific fields and types — to function correctly. An LLM, left unconstrained, produces free-form text that only sometimes happens to match that shape. This module covers the engineering discipline that closes that gap: making AI output reliable enough to be treated as software input, not prose to be re-read by a human every time.

2. Why Free-Form Text Is Hard for Software

Free-form model output:      "The ticket is about billing and
                             seems fairly urgent, maybe a 7 out of
                             10, and it probably needs escalation."

Your code needs:      {"ticket_category": "billing",
                       "urgency_score": 7,
                       "requires_escalation": true}

Parsing the free-form version reliably with regex or string matching is fragile — wording varies, and a SMALL model phrasing change can silently break your parser. Structured output exists specifically to close this gap.

3. The Complete Tool-Calling Lifecycle

1. Model DECIDES to call a tool, given its schema
2. Model GENERATES arguments matching the schema
3. Application VALIDATES those arguments (this module's focus)
4. Application EXECUTES the tool
5. Application VALIDATES the tool's OUTPUT before using it
6. Tool response is fed back to the model
7. Model CONTINUES reasoning

This directly extends your Agents course’s function-calling coverage — the engineering additions here are steps 3 and 5: validating BOTH the model’s generated arguments AND the tool’s returned output, since either one can be malformed.

4. Structured Generation — Forcing the Shape

Two approaches to getting reliably structured output:

  SCHEMA-CONSTRAINED GENERATION: the provider enforces
                                 the output shape during generation
                                 itself (many modern APIs support
                                 this directly)

  PROMPT + VALIDATE + RETRY: the prompt REQUESTS a specific JSON
                             shape, and the application VALIDATES
                             the result, retrying with an error
                             message if it doesn't match

Schema-constrained generation is more reliable when available — it prevents malformed output at the SOURCE rather than catching it after the fact. The validate-and-retry pattern remains necessary as a fallback, since not every model or provider supports constrained generation for every use case.

5. A Real-World Analogy — The Security Checkpoint

A security checkpoint doesn't just glance at a passenger and wave
them through -- it CHECKS their documentation against a REQUIRED
format (a valid passport, matching the required FIELDS: name, photo,
expiration date). A document that's close but MISSING a
field, or with a field in the WRONG format, is REJECTED,
not waved through with a shrug.

STRUCTURED OUTPUT VALIDATION is exactly this checkpoint, applied to
AI-generated data before it enters your system.

6. Fallback Strategies for Parsing Failures

When validation fails:

  1. RETRY with the validation error fed back to the model ("your
     JSON was missing field X, please regenerate")
  2. Fall back to a SIMPLER, more constrained prompt or a smaller,
     more reliable model for structured tasks specifically
  3. If retries exhaust, fail LOUDLY and route to a human
     or a default safe behavior -- never silently proceed with
     malformed data

7. A worked developer example

TechCorp’s ticket-classification pipeline:

Step	What Happens
1. Model call	Requests a JSON object: `ticket_category`, `urgency_score`, `requires_escalation`
2. Validation	Checks all three fields are present AND correctly typed
3a. If valid	Proceeds to route the ticket automatically
3b. If invalid	Retries once with the specific validation error included in the prompt
3c. If still invalid	Routes the ticket to a human for manual classification — never guesses

8. How Is This Used in the Industry?

🤖 How Is This Used in the Industry?

Production systems treat structured-output validation as a mandatory gate, not a nice-to-have — every AI-generated value that feeds into a database write, a downstream API call, or a business decision is validated first, exactly the same discipline applied to any other untrusted external input.

9. Common Mistakes

Incorrect idea: Trusting model-generated JSON without validating it.

Why it is incorrect: As shown directly in Section 5, this is equivalent to skipping input validation for any other untrusted data source.

Incorrect idea: Only validating the model’s generated arguments, not the tool’s returned output.

Why it is incorrect: As shown directly in Section 3, both directions need validation.

Incorrect idea: Silently proceeding with partially-valid or best-guess data after a validation failure.

Why it is incorrect: As shown directly in Section 6, this risks propagating bad data deeper into the system.

10. Code — A Structured Output Validator

What this shows: a working validator implementing Section 3’s step 3 — checking required fields and their types before any downstream code trusts the data, exactly Section 7’s real developer example made concrete.

import json
from dataclasses import dataclass

@dataclass
class ValidationResult:
    valid: bool
    parsed_data: dict = None
    error: str = None

def validate_structured_output(raw_output: str, required_fields: dict) -> ValidationResult:
    """A structured-output validation function -- parses,
    checks required fields AND types (Section 5's security-
    checkpoint discipline), and returns a result the rest of the
    system can safely trust."""
    try:
        data = json.loads(raw_output)
    except json.JSONDecodeError as e:
        return ValidationResult(valid=False, error=f"Invalid JSON: {e}")

    for field, expected_type in required_fields.items():
        if field not in data:
            return ValidationResult(valid=False, error=f"Missing required field: '{field}'")
        if not isinstance(data[field], expected_type):
            return ValidationResult(valid=False, error=f"Field '{field}' should be {expected_type.__name__}, "
                                                         f"got {type(data[field]).__name__}")

    return ValidationResult(valid=True, parsed_data=data)

# Exactly Section 7's schema: three required, typed fields
schema = {"ticket_category": str, "urgency_score": int, "requires_escalation": bool}

good_output = '{"ticket_category": "billing", "urgency_score": 7, "requires_escalation": true}'
malformed_output = '{"ticket_category": "billing", "urgency_score": "seven"}'

result1 = validate_structured_output(good_output, schema)
result2 = validate_structured_output(malformed_output, schema)

print(f"Valid output: {result1.valid}, data: {result1.parsed_data}")
print(f"Malformed output: {result2.valid}, error: {result2.error}")

Expected Output:

Valid output: True, data: {'ticket_category': 'billing',
'urgency_score': 7, 'requires_escalation': True}
Malformed output: False, error: Field 'urgency_score' should be int,
got str

What this confirms: well-formed output passes and is returned as trustworthy, parsed data, while the malformed output (a string where an integer was required) is correctly rejected with a specific, actionable error — exactly the kind of error message Section 6’s retry strategy would feed back to the model for correction, rather than silently accepting the bad value.

11. Production Considerations

Validation error messages fed back to the model for a retry (Section 6) should be specific enough for the model to fix the exact problem, not a generic “invalid output” message
Log every validation failure (Module 12) — a rising failure rate for a specific field often signals a prompt or schema problem worth investigating

12. Trade-offs

Retry-on-failure adds latency to the failure path — bounded retries (Section 6) balance reliability against not stalling indefinitely
Schema-constrained generation (Section 4), when available, is more reliable but may not be supported by every model or provider you need to use

13. Chapter Summary

Structured output engineering is what turns unreliable free-form model text into software your system can safely trust — through schema-constrained generation where available, and rigorous validate-then-retry discipline everywhere else. The complete tool-calling lifecycle requires validation in BOTH directions: the model’s generated arguments before execution, and the tool’s returned results before the model reasons over them.

A validation failure should never be silently ignored — retry with a specific error, or fail loudly and route to a human.

14. Visual Cheat Sheet

Model generates arguments -> VALIDATE -> execute tool
Tool returns output       -> VALIDATE -> feed back to model

Validation fails -> retry with SPECIFIC error -> still fails ->
                     fail LOUDLY, route to human (never guess)

15. Top Takeaways

Free-form model text is unreliable software input — structured output validation closes that gap.
The tool-calling lifecycle needs validation in BOTH directions: generated arguments AND returned tool output.
Schema-constrained generation, where available, is more reliable than prompt-request-and-hope.
A validation failure should trigger a specific, actionable retry or a loud failure — never silent, best-guess proceeding.
Treat AI-generated structured data with the same input-validation discipline as any other untrusted external input.

16. Interview Questions

Q: 1. Why is validating model-generated JSON as important as validating any other untrusted input?**

Ans: Model output, like user input or third-party API responses, is not guaranteed to match your expected shape — it can be malformed, missing fields, or have incorrectly-typed values.

Treating it as automatically trustworthy risks propagating bad data into database writes, downstream API calls, or business decisions, exactly the same risk as skipping validation on any other external data source.

Why it matters: This is a common source of production bugs when teams treat “the model said so” as inherently reliable.
Real-world example: Section 10’s malformed-output example — a string where an integer was required would cause a downstream type error if not caught first.
Common mistake: Parsing model output with json.loads() and using it directly with no field or type checks.
Interviewer is testing: Whether the candidate applies standard input-validation discipline to AI-generated data, not just traditional user input.
Likely follow-up: “What would you do on a validation failure?” → Retry with the specific error fed back to the model, then fail loudly and route to a human if retries are exhausted (Section 6).

Q: 2. What’s the difference between schema-constrained generation and a validate-then-retry approach, and when would you use each?**

Ans: Schema-constrained generation has the provider enforce the output shape during generation itself, preventing malformed output at the source. Validate-then-retry requests a shape via the prompt and checks the result afterward, retrying with an error message if it doesn’t match.

Constrained generation is more reliable when available; validate-then-retry remains necessary as a fallback for cases or providers where constrained generation isn’t supported.

Why it matters: Relying purely on prompt wording (“please return valid JSON”) without either enforcement mechanism produces unreliable results at production scale.
Real-world example: A team using a model/provider combination that supports schema-constrained generation for structured classification tasks would prefer that route over prompt-and-hope.
Common mistake: Assuming a well-worded prompt alone reliably guarantees valid JSON without any enforcement or validation mechanism.
Interviewer is testing: Whether the candidate knows concrete, reliability mechanisms beyond prompt wording alone.
Likely follow-up: “How would you handle a model/provider that doesn’t support constrained generation?” → Validate-then-retry (Section 6), with bounded retries and a loud failure path.

17. Scenario-Based Question

Scenario: TechCorp’s ticket-classification pipeline works correctly 95% of the time. In the remaining 5%, the pipeline throws an unhandled exception, crashing the request, because the model occasionally omits the requires_escalation field or returns it as the string "true" instead of a boolean.

Problem Analysis: Section 9’s common mistake — no structured-output validation gate before the data is used.
How to Think: The 5% failure rate isn’t a model-quality problem to fix with prompt tweaking alone — it’s a missing engineering safeguard that should exist regardless of how rare the malformed case is.
Investigation: Confirm the exact malformed output patterns (missing field, wrong type) by reviewing crash logs.
Root Cause: No validation layer between the model’s raw output and the code that uses it — the pipeline assumes well-formed JSON unconditionally.
Solution: Add Section 10’s validation function as a mandatory gate; on failure, retry once with the specific error (Section 6), then route to a human if still invalid, rather than crashing.
Trade-offs: Adds a small amount of latency to every request for the validation check itself — worthwhile given the alternative is unhandled crashes reaching real users.
Production Considerations: This is exactly the kind of gap Section 8’s worked developer example is designed to prevent — the validation gate should exist from the start, not be added reactively after a crash-rate investigation.

18. Next Step

Next: Module 10 — AI Evaluation Deep Dive — Level 4 begins here: what it means for an AI system to be “correct,” golden datasets, LLM-as-judge, and building a complete evaluation pipeline.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026