AI FOUNDATIONS
Transformers for AI
Understand the architecture behind modern language models, generative AI and multimodal systems โ from attention and positional encoding to the full Transformer block and modern LLM inference.
Transformers Learning Path
Learn the concepts in order. Each module builds toward a deeper understanding of how attention-based models scale into modern AI systems.
Why Transformers?
Understand the specific technical problems with RNNs and LSTMs that Transformers solved, and trace the historical progression from RNN to attention to Transformer to modern LLMs โ with a concrete demonstration of why attention parallelizes and recurrence doesn't.
Transformer Input: From Text to Vectors
Trace the complete path from raw text to what actually enters a Transformer โ tokenization, token IDs, embedding lookup, and positional information โ with a fully verified numeric example showing why identical tokens need positional information to be distinguished.
Self-Attention From First Principles
Build self-attention from raw intuition, before ever seeing the formula โ a token deciding which other tokens are relevant to it, computed with real numbers on a tiny 4-token example, verified step by step.
Query, Key and Value
Understand why real Transformers use three separate learned projections โ Query, Key, and Value โ instead of raw embeddings directly, with a fully verified hand-worked example showing each token gets three genuinely different vectors.
Scaled Dot-Product Attention
Understand the formal attention equation โ softmax(QKแต/โdโ)V โ and precisely why the โdโ scaling factor is mathematically necessary, with a verified demonstration of softmax saturation at high dimensions with and without scaling.
Attention Masks and Causal Attention
Understand padding masks and causal masks, why future tokens must be hidden during autoregressive generation, and how upper-triangular masking with -infinity before softmax makes GPT-style generation work โ with a fully verified example.
Multi-Head Attention
Understand why a single attention computation isn't enough, how multiple attention heads run in parallel on different learned subspaces, and how their outputs are concatenated and projected โ with a fully verified two-head example.
Positional Information
Understand why attention alone cannot distinguish word order, and how sinusoidal positional encoding, learned positional embeddings, and RoPE solve this โ with a direct, verified proof using 'dog bites man' vs 'man bites dog'.
The Transformer Block
Assemble multi-head attention, residual connections, layer normalization, and the feed-forward network into the complete, repeating unit every Transformer-based LLM is built from โ with a fully verified end-to-end computation.
Feed Forward Networks
Understand the position-wise feed-forward network inside every Transformer block โ expansion, activation, contraction โ and why it transforms each token independently, with a direct proof and modern gated variants like SwiGLU.
Residual Connections and Layer Normalization
A focused, Transformer-specific look at residual connections and LayerNorm โ why they're essential for training deep stacks of blocks, and the real, verified difference between Pre-LN and Post-LN architectures.
The Complete Transformer Architecture
Assemble the full Transformer architecture โ from text to a stack of Transformer blocks producing a final representation โ and get an overview of encoder-only, decoder-only, and encoder-decoder model families.
Encoder vs Decoder vs Decoder-Only LLMs
A deeper look at encoder, decoder, and encoder-decoder architectural tradeoffs, with a verified cross-attention computation, and precisely why modern LLMs overwhelmingly use decoder-only architectures.
How GPT-Style LLMs Actually Work
The major integration module โ trace one complete request, 'Explain RAG', all the way from tokenization through causal decoder blocks to a predicted next token, using nothing but mechanisms already built and verified in this course.
Training a Transformer
Understand how a Transformer is trained โ shifted tokens, teacher forcing, and cross-entropy loss computed at every position simultaneously โ connecting directly to the loss, backpropagation, and optimizer mechanics from the Deep Learning course.
Transformer Inference and KV Cache
Understand why autoregressive inference recomputes redundant work without caching, how the KV cache eliminates it, and the prefill/decode phases โ with a verified proof of both correctness and a real efficiency gain.
Transformer Scaling, Context Windows and Efficiency
Understand why attention's computational cost scales quadratically with sequence length, why this makes long-context LLM inference expensive, and how techniques like grouped-query and multi-query attention reduce that cost โ with verified numbers.
Transformers โ LLMs โ Modern AI Systems
The final integration module โ map the complete evolution from Transformers to modern AI systems, precisely distinguish Transformer, LLM, Generative AI, and Agentic AI, and establish the bridge into the dedicated LLM course.
WHY TRANSFORMERS?
Learn Transformers through the lens of modern AI
Transformers replaced sequential recurrence with attention and parallelism, making it possible to train massive models that can understand and generate language at scale. This is the architecture behind modern LLMs, chat systems, retrieval, summarization and generative AI workflows.