LLM Applications in AI Engineering
Every LLM Applications lesson in the AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons and their workbooks are in the plan.
Calling models well: JSON schema outputs, validation and retry
Calling LLMs in real software: getting JSON back instead of prose, validating the output and retrying, measured on invoice extraction with a small model.
Retrieval: chunking, embeddings and hybrid keyword search
RAG retrieval explained: why retrieve, measuring retrieval and generation separately, keyword search in the BM25 style, and ways to make retrieval better.
Evaluating LLM apps: eval sets, rule-based checks and LLM-as-judge
How to evaluate LLM apps: build an evaluation set from real usage, grade with rules or an AI judge, and run evals to catch regressions before users do.
Prompt injection and data poisoning: layered guardrails
Prompt injection and data poisoning in LLM apps: an instruction planted in a note, false facts that look like data, and guardrails layered in defence.
Cost, latency and model routing: token economics and batching
Where an LLM app's cost and latency go: input versus output tokens, small versus big models, batching, and the levers that lower the bill.
LLM Tracing and Debugging with OpenTelemetry
Page coming soon
Agents in production: workflow versus agent loop, cost and tool reach
AI agents in production: when to use a fixed workflow instead of an agent, what the agent loop costs, what it can reach, and how to ship one safely.
Build a RAG Assistant: Project with Evals
Page coming soon
