LLM Systems in AI Engineering
Every LLM Systems lesson in the AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons and their workbooks are in the plan.
LLM inference: sampling strategies, prefill and decode, KV cache, quantization, speculative decoding and test-time compute
How LLMs generate text and how to make it fast: greedy, top-k and top-p sampling, prefill and decode, the KV cache, quantization and speculative decoding.
Retrieval-augmented generation, BM25 and HNSW search, tool use and coding agents
Why a language model needs retrieval and tools: RAG, BM25 and nearest-neighbour search, tool calls, the agent loop, context engineering and coding agents.
Multimodal models: ViT patches, CLIP contrastive learning, CTC speech recognition and DDPM diffusion
How one transformer handles images, speech and pictures: ViT patches, CLIP's shared space, vision-language models, speech to text with CTC, and diffusion.
Capstone: an end-to-end RAG agent with a trained reranker, multimodal input, red-teaming and deployment
A capstone project: an AI system around a model you trained, with retrieval, an agent loop, a multimodal channel, evaluation, red-teaming and deployment.
