Section 2 — Prompt API

Begin with the problem

Turning messages into a model request

A model does not receive a Java method call such as “answer this.” It receives an ordered set of messages, instructions, variables, and options. The Prompt API builds that package.

template + variables + roles + options → Prompt

What you will learn

• Build a Prompt from system and user messages.
• Explain why message roles matter.
• Use templates without mixing trusted instructions and untrusted text.
• Know when a plain string is enough and when Prompt is useful.

Current official reference: Spring AI documentation for this topic. The examples below primarily preserve the stated 1.1.x course target. Where Spring AI 2.0 differs, the text must treat that behavior as version-specific rather than universal.

(Continues from Section 1. Same target: Spring AI 1.1.x / Spring Boot 3.5.x.)

2.1 Why the Prompt API Exists as a Separate Layer

Beginner primer — prompt engineering basics: if you haven’t written prompts before, the short version is: system/user/assistant roles are how you structure a conversation (see the glossary if this is new). Few-shot prompting means showing the model a couple of example question/answer pairs before your real question, so it learns the expected style or format by imitation rather than instruction alone. Chain-of-thought prompting means asking the model to reason step-by-step before giving a final answer, which measurably improves accuracy on multi-step problems. None of these are Spring AI features — they’re general LLM prompting techniques that predate Spring AI entirely. What Spring AI adds is not new prompting theory; it’s a typed, composable, testable object model around text that would otherwise be raw string concatenation scattered across your codebase.

The problem Spring AI is solving here is the same one JPA solved for SQL: without an abstraction, every service method builds prompt strings with ad hoc String.format or StringBuilder calls, template variables get injected inconsistently (some escaped, some not), and there’s no single place to unit-test “does this prompt render correctly for these inputs” independent of an actual model call. Prompt + PromptTemplate give you that seam.

Real-world analogy — Bank Loan Letter Templates: A bank doesn’t have loan officers freehand-write approval letters. They have a fixed template with placeholders ({{customerName}}, {{loanAmount}}, {{interestRate}}) validated and filled by a system before a human ever sees the letter. PromptTemplate is that system: the placeholders are {variableName} (StringTemplate/Mustache-style by default, actually backed by Apache StringTemplate under the hood — more on that in §2.6), and PromptTemplate.render(Map<String,Object>) is the letter-printing step. You never let free-form string concatenation reach the “customer” (the model).

Analogy: The Corporate Form Letter (Mail Merge) Imagine a corporate bank processing loan approval notifications:

• The Problem: You don’t let bank branch managers type out custom, freeform emails from scratch to customers. They might make typos, forget legal disclosures, or write inappropriate conditions.
• The Template (PromptTemplate): The compliance department designs a standard form letter with strict placeholders: "Dear {customerName}, your loan of {amount} at {interest}% interest is approved." (This is Apache StringTemplate ST4 format).
• The Rendering (Map Merge): When a manager hits “Send”, a software program automatically pulls data fields from the client database, merges them into the template variables, and formats a clean, official PDF notification (a rendered UserMessage / Prompt).
• The final prompt goes out containing system rules, conversation history, and the user turn, wrapped safely inside the envelope (Prompt).

📊 Visual Chart: Prompt Object Model & Message Hierarchy

Here is the class layout representing instructions, chat options, and roles passed to the model client:

classDiagram
    class Prompt {
        +List~Message~ messages
        +ChatOptions chatOptions
    }
    class Message {
        <<interface>>
        +MessageType getMessageType()
        +String getText()
        +Map~String,Object~ getMetadata()
    }
    class SystemMessage {
        +MessageType SYSTEM
    }
    class UserMessage {
        +MessageType USER
        +List~Media~ media
    }
    class AssistantMessage {
        +MessageType ASSISTANT
        +List~ToolCall~ toolCalls
    }
    class ToolResponseMessage {
        +MessageType TOOL
    }

    Prompt --> Message : contains
    Message <|-- SystemMessage : implements
    Message <|-- UserMessage : implements
    Message <|-- AssistantMessage : implements
    Message <|-- ToolResponseMessage : implements

---

2.2 The Object Model

┌───────────────────────────────────────────────────────────────┐
│                            Prompt                                │
│  - List<Message> instructions                                    │
│  - ChatOptions chatOptions                                       │
│  (immutable; this is literally what gets sent to ChatModel)      │
└───────────────────────────────┬─────────────────────────────────┘
                                 │ composed of
                                 ▼
┌───────────────────────────────────────────────────────────────┐
│                     Message (interface)                          │
│     MessageType getMessageType()                                 │
│     String getText()                                             │
│     Map<String,Object> getMetadata()                             │
└───────────────────────────────┬─────────────────────────────────┘
             ┌───────────────────┼───────────────────┬─────────────────┐
             ▼                   ▼                   ▼                 ▼
    SystemMessage         UserMessage        AssistantMessage   ToolResponseMessage
  (MessageType.SYSTEM)  (MessageType.USER)  (MessageType.ASSISTANT)  (MessageType.TOOL)
                                 │
                        also carries:
                        List<Media> media (images/audio for multimodal input)

AssistantMessage is special: it’s the only message type that can carry List<AssistantMessage.ToolCall> — the parsed tool-invocation requests the model returned. This is how tool-calling round-trips are represented: the model responds with an AssistantMessage containing tool calls instead of text, your code executes them, wraps results in ToolResponseMessage, and both get appended to the running List<Message> for the next round. (Tool calling itself is covered fully in Section 9; the point here is just how it’s represented in the message model.)

---

2.3 PromptTemplate — Internal Rendering Pipeline

PromptTemplate template = new PromptTemplate("""
        You are a {role} for {company}.
        Answer the following customer question using only the
        provided context. If the answer isn't in the context, say so.

        Context:
        {context}

        Question:
        {question}
        """);

Prompt prompt = template.create(Map.of(
        "role", "senior support engineer",
        "company", "Acme Corp",
        "context", retrievedContext,
        "question", userQuestion
));

2.3.1 What Happens Inside .create()

PromptTemplate.create(Map<String,Object> variables)
        │
        ▼
1. TemplateRenderer resolves — by default StTemplateRenderer
   (wraps Apache StringTemplate 4 / "ST4"), NOT simple String.replace.
   This matters: ST4 gives you conditional blocks, iteration, and
   strict validation, at the cost of needing to escape literal
   curly braces "\{" if your prompt text legitimately contains "{}"
   (e.g., you're asking the model to emit JSON — this bites people
   constantly, see §2.9 common mistakes)
        │
        ▼
2. TemplateRenderer.apply(template, variables) → rendered String
        │
        ▼
3. Rendered String wrapped as new UserMessage(rendered)
        │
        ▼
4. new Prompt(List.of(userMessage)) — or, if you used
   PromptTemplate.createMessage(), you get just the Message,
   letting you compose it into a larger List<Message> yourself

2.3.2 PromptTemplate vs. SystemPromptTemplate vs. Prompt.builder()

Class	Produces	Typical use
PromptTemplate	Renders to a UserMessage by default via .create()	Single user-turn templated prompts
SystemPromptTemplate	Renders to a SystemMessage	Reusable system instructions with variables (e.g., persona name, tone, domain) injected per-tenant
Prompt.builder() (or new Prompt(List<Message>...))	Assembles multiple already-rendered messages + ChatOptions into the final immutable Prompt	Composing system + few-shot examples + user turn into one call

A realistic multi-part assembly:

SystemMessage system = new SystemPromptTemplate(systemTemplateResource)
        .createMessage(Map.of("tenantName", tenant.getName(), "tone", tenant.getTone()));

List<Message> fewShotExamples = List.of(
        new UserMessage("Refund a $20 order"),
        new AssistantMessage("I've processed a $20 refund. It'll appear in 3-5 business days."),
        new UserMessage("Cancel my subscription"),
        new AssistantMessage("Your subscription is canceled effective end of billing period.")
);

UserMessage userTurn = new PromptTemplate(userTemplateResource)
        .createMessage(Map.of("question", incomingQuestion));

List<Message> allMessages = new ArrayList<>();
allMessages.add(system);
allMessages.addAll(fewShotExamples);
allMessages.add(userTurn);

Prompt prompt = new Prompt(allMessages, OpenAiChatOptions.builder()
        .model("gpt-4o")
        .temperature(0.2)
        .build());

This is the actual internal shape of “few-shot prompting inside Spring AI” — there is no special FewShotPromptTemplate class; few-shot is just alternating UserMessage/AssistantMessage pairs assembled into the List<Message> before the real user turn. Chain-of-thought is the same story: it’s a system-prompt instruction (“think step by step before answering”) or a few-shot example demonstrating the reasoning trace — Spring AI provides no special API surface for it because it doesn’t need one; it’s pure prompt content.

---

2.4 Loading Templates From Resources (Production Pattern)

Hardcoding multi-line prompt text as Java text blocks works for a demo; production teams externalize templates so they can be versioned, reviewed, and hot-swapped without a redeploy in some setups.

# application.yml
app:
  prompts:
    support-system: classpath:/prompts/support-system.st
    support-user: classpath:/prompts/support-user.st

@Configuration
public class PromptTemplateConfig {

    @Value("classpath:/prompts/support-system.st")
    private Resource supportSystemResource;

    @Bean
    public SystemPromptTemplate supportSystemTemplate() {
        return new SystemPromptTemplate(supportSystemResource);
    }
}

src/main/resources/prompts/support-system.st:

You are {role}, the AI support assistant for {company}.

Rules:
- Never invent policy details not present in the provided context.
- If uncertain, escalate rather than guess.
- Tone: {tone}

Why the .st extension, not .txt: Apache StringTemplate’s default STGroupFile convention expects .st; Spring AI’s StTemplateRenderer doesn’t strictly require the extension when you load via Resource, but the convention signals intent to every engineer who opens the file, and IDE plugins for ST4 syntax highlighting key off it.

---

2.5 Structured Prompt Content — Multimodal Messages

A UserMessage isn’t limited to text. For multimodal input (vision models — models that can accept an image alongside text and reason about both together):

UserMessage multimodalMessage = UserMessage.builder()
        .text("What's defective in this product photo? Reference our QA checklist.")
        .media(new Media(MimeTypeUtils.IMAGE_PNG, imageResource))
        .build();

Internally, OpenAiChatModel (and any other multimodal-capable model implementation) inspects Message.getMedia() and, per-provider, base64-encodes the bytes or references a URL depending on what the provider’s wire format expects — this transformation lives entirely inside the provider module (OpenAiApi.ChatCompletionMessage construction), invisible to your service code. This is the portability layer earning its keep: you write one Media object; each provider module decides how to serialize it.

---

2.6 Internal Working — Why StringTemplate (ST4), Not Simple Interpolation

This is a detail almost nobody reads the source for, and it explains a very common bug. Spring AI’s default TemplateRenderer is StTemplateRenderer, backed by Apache StringTemplate 4 (ST4), not naive "{var}".replace(). ST4 was chosen because it supports:

• Strict undefined-variable detection (fails loudly instead of silently leaving {typo} in the output)
• Conditional rendering (<if(flag)>...<endif>) for prompts that vary structurally by input, not just by substituted values
• Iteration over lists for building numbered context blocks

The bug this causes: if your prompt template legitimately needs literal curly braces — extremely common when you’re instructing the model to output JSON —

// BROKEN: ST4 tries to interpret {"name": "..."} as template syntax
new PromptTemplate("Respond ONLY with JSON like {\"status\": \"ok\"}");

you must either escape the braces or switch renderers:

// Option 1: escape literal braces for ST4
new PromptTemplate("Respond ONLY with JSON like \\{\"status\": \"ok\"\\}");

// Option 2: swap the TemplateRenderer entirely for templates with heavy
// literal-brace content — configure a custom delimiter instead of { }:
PromptTemplate template = PromptTemplate.builder()
        .renderer(StTemplateRenderer.builder()
                .startDelimiterToken('<')
                .endDelimiterToken('>')
                .build())
        .template("Respond ONLY with JSON like {\"status\": \"<status>\"}")
        .build();

This is one of the highest-signal “internal working” facts in this whole section: know that Spring AI’s prompt rendering is not naive string substitution, and budget for delimiter conflicts whenever your prompts describe JSON output shapes — which, given Section 11 — Structured Output, is most of them in production.

---

2.7 Reusable Prompt Libraries — Production Pattern

At scale, teams centralize prompt templates the same way they centralize SQL — a PromptRegistry bean, or externalized .st resources under version control, sometimes backed by a database table for A/B-tested prompt versions.

@Service
public class PromptRegistry {

    private final Map<String, PromptTemplate> templates;

    public PromptRegistry(ResourcePatternResolver resolver) throws IOException {
        this.templates = new HashMap<>();
        Resource[] resources = resolver.getResources("classpath:/prompts/*.st");
        for (Resource r : resources) {
            String key = r.getFilename().replace(".st", "");
            templates.put(key, new PromptTemplate(r));
        }
    }

    public Prompt render(String templateName, Map<String, Object> vars) {
        PromptTemplate t = templates.get(templateName);
        if (t == null) {
            throw new IllegalArgumentException("Unknown prompt template: " + templateName);
        }
        return t.create(vars);
    }
}

Pair this with Section 13’s observability guidance: log the rendered prompt (not the raw template) with a promptTemplateVersion tag on your Micrometer observation so you can correlate model-quality regressions with a specific prompt deployment — this is the foundation of “prompt versioning,” part of the broader discipline sometimes called LLMOps (the AI-specific extension of DevOps/MLOps practices — versioning prompts, tracking model-quality regressions, and treating prompt changes with the same rigor as code changes).

---

2.8 Lifecycle Summary

Template authored (.st resource or text block)
        │
        ▼
PromptTemplate instantiated (once, typically as a singleton bean
or cached in a registry — rendering is stateless, the template
object itself is immutable and reusable across calls)
        │
        ▼
.create(vars) / .createMessage(vars) called PER REQUEST
        │
        ▼
Rendered Message(s) composed into Prompt
        │
        ▼
Prompt is IMMUTABLE from here — passed to ChatClient/ChatModel,
never mutated after construction
        │
        ▼
Advisors may produce a NEW Prompt (e.g., memory advisor prepends
history) — they don't mutate the original in place

Key point: PromptTemplate instances are safe to share as singleton beans — they hold no per-request state. Prompt instances are not meant to be reused across unrelated calls; build a new one per request (or let ChatClient build it for you internally from your .user()/.system() builder calls, covered in Section 3).

---

2.9 Common Mistakes

1. Forgetting to escape literal {} when prompting for JSON output — causes STException/AttributeRenderer errors, or worse, silently-wrong rendering depending on ST4 version behavior.
2. Rebuilding PromptTemplate on every request instead of caching it — wasteful (though ST4 parsing is fast, it’s still unnecessary allocation at high QPS) and makes prompt-version tracking harder since there’s no stable object to tag.
3. Concatenating retrieved RAG context directly into the user question string instead of using a distinct {context} variable — makes prompt-injection auditing and context-length debugging much harder (see Section 15 — Security).
4. Not validating variable maps before .create() — a missing key throws at render time, not at compile time; production code should validate required keys upfront rather than letting a STException surface as a generic 500.
5. Treating system prompts as static strings instead of templates — hardcoded tone/persona/company name makes multi-tenant SaaS prompt customization impossible without a redeploy.

---

2.10 Debugging

Enable ST4-level tracing when a render doesn’t produce what you expect:

logging:
  level:
    org.springframework.ai.chat.prompt: DEBUG

For a fast local check outside the full Spring context:

PromptTemplate t = new PromptTemplate(myTemplateString);
System.out.println(t.render(myVarsMap)); // inspect rendered output before it ever hits a model

This single-line check — render without calling the model — should be your first debugging step whenever “the model gave a weird answer” turns out to actually be “the prompt didn’t render the way I assumed.”

---

2.11 Interview Questions

1. What templating engine actually powers PromptTemplate by default, and why was it chosen over naive string substitution?
2. How does Spring AI represent few-shot examples internally — is there a dedicated FewShotPromptTemplate class?
3. Why does prompting for JSON output frequently break with PromptTemplate, and what are the two ways to fix it?
4. What’s the difference between PromptTemplate.create() and PromptTemplate.createMessage()?
5. Is Prompt mutable or immutable? What are the implications for how Advisors modify a request mid-pipeline?
6. How does SystemPromptTemplate differ from PromptTemplate at the type level, and why does that distinction matter for message ordering?
7. Where does chain-of-thought prompting “live” in Spring AI’s API surface?
8. How would you externalize prompt templates for a multi-tenant SaaS product where each tenant needs custom persona/tone?
9. What class handles multimodal Media attachment on a UserMessage, and where does provider-specific encoding actually happen?
10. Why is caching PromptTemplate instances as singleton beans safe, while reusing a Prompt instance across unrelated requests is not?
11. What’s the fastest way to debug “the model got a weird prompt” without making an actual API call?
12. How would you implement prompt-version tracking for A/B testing prompt changes in production?
13. What exception type surfaces when a required template variable is missing at render time, and how would you validate against this proactively?
14. Explain how List<Message> ordering (system → few-shot pairs → user) affects model behavior, and where that ordering is actually assembled in the codebase.
15. What custom delimiter configuration would you use for a template whose content is dominated by literal JSON braces?
16. How do ToolResponseMessage and AssistantMessage.ToolCall fit into the Message hierarchy, and why aren’t they just variants of UserMessage/AssistantMessage text?
17. What’s the production risk of concatenating RAG-retrieved content directly into the user’s raw question string instead of a separate template variable?
18. How would you unit test a PromptTemplate render without spinning up a Spring context or hitting a model API?
19. What’s the relationship between MessageType and the Message interface hierarchy?
20. Describe the full assembly path from “developer writes a .st file” to “bytes sent over HTTP to the model provider.”

---

2.12 Best Practices Checklist

• Externalize all non-trivial prompts as versioned .st resources, not inline text blocks, once past prototype stage.
• Escape or reconfigure delimiters for any template instructing JSON output.
• Cache PromptTemplate instances as singleton beans or in a PromptRegistry; never rebuild per request.
• Log rendered (not raw template) prompts with a version tag for observability correlation.
• Keep retrieved RAG context in a dedicated template variable, never string-concatenated into the raw user question.
• Validate required template variables before calling .create() to turn missing-key failures into clear domain errors, not opaque STExceptions.

---

2.13 Key Takeaways

• PromptTemplate is backed by Apache StringTemplate (ST4), not naive interpolation — this has real consequences for JSON-shaped prompts.
• Few-shot and chain-of-thought have no dedicated API; they’re plain Message list composition — Spring AI intentionally adds no ceremony here.
• Prompt is immutable; Advisors produce new Prompts rather than mutating in place.
• Multimodal content flows through Media on UserMessage, with provider-specific encoding isolated inside each provider module.
• Treat prompts as versioned, testable artifacts — render-without-calling-the-model is your cheapest debugging tool.

---

End of Section 2. Next: Section 3 — ChatClient (Builder, lifecycle, streaming vs. blocking, retry/timeout, interceptors, request/response pipeline, memory, execution flow).