Local AI Workloads on Linux

Every Local AI Workloads on Linux lesson in the Linux for AI Engineering course, in the order you study them. Each page opens about the first half of its lesson to read, some the whole of it; the full lessons are in the plan.

3 lessons

  1. Local LLM inference: memory bandwidth, model files and quantization, llama.cpp, Ollama and vLLM

    Running LLMs locally on Linux: why a model that spills out of GPU memory is so slow, tokens per second, quantization, llama.cpp and Ollama.

  2. Voice and vision workloads: latency budgets, PipeWire and ALSA, FFmpeg and headless EGL graphics

    Local voice and vision AI on Linux: why a voice assistant feels slow, the latency budget, the Linux audio stack, FFmpeg and speech models.

  3. Local AI applications: RAG pipelines, chunking, headless browsers and always-on agents

    Building local AI apps on Linux: why a RAG app misses answers in a PDF, OCR, chunking, embeddings and vector stores, and browsers.