Machine Learning Foundations in AI Engineering
Every Machine Learning Foundations lesson in the AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons and their workbooks are in the plan.
Bigram language models: count-based estimation, sampling, smoothing and perplexity
How a bigram language model predicts the next word from counted pairs, how it generates text by sampling, and how perplexity judges it.
Distributional semantics: one-hot encoding, co-occurrence matrices, PCA and SVD
Vectors, dot products and cosine similarity, one-hot word vectors, meaning from co-occurrence, and how PCA and SVD compress words into dense vectors.
Probability, entropy and cross-entropy, and gradient descent on computational graphs
Joint, marginal and conditional probability from counts, entropy and cross-entropy loss, and how gradient descent minimises it through a computational graph.
