Local LLMs in AI Engineering
Every Local LLMs lesson in the AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons and their workbooks are in the plan.
LLM Memory Requirements: How Much RAM You Need
Page coming soon
Quantization: 16-bit, 8-bit and 4-bit weights, measured
LLM quantization explained: rounding weights onto a grid of 256 or 16 values, in groups with their own scale, and what 16, 8 and 4 bits do to quality.
How to Run an LLM Locally: llama.cpp and Ollama
Page coming soon
LLM Tokens per Second: Why Memory Bandwidth Wins
Page coming soon
Private retrieval-augmented generation over your own files
Build a private assistant over your own files: retrieval on your machine, so documents never leave your disk, measured with and without your notes.
Build an Offline AI Assistant: Project
Page coming soon
