Local LLMs in AI Engineering

Every Local LLMs lesson in the AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons and their workbooks are in the plan.

6 lessons

  1. LLM Memory Requirements: How Much RAM You Need

    Page coming soon

  2. Quantization: 16-bit, 8-bit and 4-bit weights, measured

    LLM quantization explained: rounding weights onto a grid of 256 or 16 values, in groups with their own scale, and what 16, 8 and 4 bits do to quality.

  3. How to Run an LLM Locally: llama.cpp and Ollama

    Page coming soon

  4. LLM Tokens per Second: Why Memory Bandwidth Wins

    Page coming soon

  5. Private retrieval-augmented generation over your own files

    Build a private assistant over your own files: retrieval on your machine, so documents never leave your disk, measured with and without your notes.

  6. Build an Offline AI Assistant: Project

    Page coming soon