The myth of 1-bit LLMs | Extreme Quantization

Are 1-bit LLMs the future of efficient AI? Or just a catchy Microsoft metaphor? In this video, we break down BitNet, the so-called “1-bit LLM” that isn’t really 1-bit—but still delivers massive speed and memory gains through extreme quantization. 🔍 What you’ll learn: • What fractional (1.58) bits are • How BitNet works under the hood (BitLinear, ELUT, TL1/TL2) • The role of quantization-aware training (QAT) and the Straight-Through Estimator (STE) • Optimizations for ternary matrix multiplication • How 1-bit LLMs scale with parameter count 📄 Main paper: https://arxiv.org/abs/2402.17764 Get my full paper reading list from here 👉 https://www.patreon.com/posts/130059217 👉 Watch the full Model Quantization Series: https://youtube.com/playlist?list=PL4bm2lr9UVG0HvePBXvsceO4yuLC8HhUh&si=Wd5vK6B2HQNAL67J 00:00 Intro 01:05 Inspiration and motivation 05:20 BitNet model architecture 10:21 Quantization-Aware Training 15:21 Storing fractional bits: bitpacking & ELUT 18:12 Open-weights models in HuggingFace 19:52 Ternary matrix multiplication 21:20 Demo & evaluation 23:59 Outro