AI Systems
The same discipline, a different cost structure.
Everything from the classical curriculum still applies — caching, queues, timeouts, backpressure, failure isolation. What changes is where the money and the milliseconds go. Latency scales with output length. Spend scales with a context window that grows on every loop iteration. And the slowest dependency in your system is one you do not operate.
LLM Request Trace
Follow an agentic request through tool selection, an MCP server, and final synthesis. Change the model tier or add a tool call and watch what it does to latency and cost.
- Client18ms
Request leaves the browser.
- API Gateway12ms
Auth, rate limiting, request validation.
- Agent loop6ms
Assembles the system prompt, tool schemas, and conversation history.
- Model call 1 — tool selection1.27s
Decides which tool to call and with what arguments.
in 4K tokout 90 tok$0.013 - MCP server — tool execution180ms
Runs the tool and returns a result the model has to read back.
- Model call — final response3.36s
Generates 320 output tokens at roughly 9ms each.
in 4.5K tokout 320 tok$0.018 - Streaming response10ms
Tokens are forwarded as they are produced.
With streaming, the user sees output after the first token, not after the last. Time to first token is the number that governs perceived speed.
The usual default. Handles retrieval and multi-step tool use reliably.
Each call is another full model round trip, with the growing transcript resent every time.
Latency scales with output length. Input is processed in parallel; output is generated one token at a time.
Model tiers are generic and prices are illustrative, chosen to show the shape of the trade-off rather than any specific provider’s rates.
Curriculum
Seven complete lessons now cover the linked topics, including the whole of LLM application architecture. The remaining roadmap topics are still in development.
LLM Application Architecture
- LLM Request LifecycleLesson →
- Streaming & Time to First TokenLesson →
- Structured OutputsLesson →
- Model RoutingLesson →
- Token & Cost ManagementLesson →
- Model FallbackLesson →
- AI GatewaysLesson →
Retrieval
- EmbeddingsLesson →
- Vector SearchLesson →
- Chunking StrategyLesson →
- RAG PipelinesLesson →
- RerankingLesson →
- Semantic CachingLesson →
Agentic Systems
- Tool CallingLesson →
- Agent LoopsLesson →
- MCPLesson →
- Sandboxing
- Human ApprovalLesson →
- Agent MemoryLesson →
- CheckpointingLesson →
- Long-Running WorkflowsLesson →
- Multi-Agent Systems
Operating AI Systems
- AI Observability
- Evals
- Guardrails
- Failure Recovery
- Provider Rate Limits