Machine Learning Foundations in AI Engineering

Every Machine Learning Foundations lesson in the AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons and their workbooks are in the plan.

3 lessons

  1. Bigram language models: count-based estimation, sampling, smoothing and perplexity

    How a bigram language model predicts the next word from counted pairs, how it generates text by sampling, and how perplexity judges it.

  2. Distributional semantics: one-hot encoding, co-occurrence matrices, PCA and SVD

    Vectors, dot products and cosine similarity, one-hot word vectors, meaning from co-occurrence, and how PCA and SVD compress words into dense vectors.

  3. Probability, entropy and cross-entropy, and gradient descent on computational graphs

    Joint, marginal and conditional probability from counts, entropy and cross-entropy loss, and how gradient descent minimises it through a computational graph.