Rotary Positional Encodings | Explained Visually

In this lecture, we learn about Rotary Positional Encodings (RoPE). This is the type of positional encoding used by most modern LLMs. It sets the stage to understand how DeepSeek modified the Multi-head Latent Attention (MLA) mechanism with RoPE. Everything is explained from scratch, on a whiteboard. ====================================================== This video is sponsored by invideoAI (https://invideo.io/). invideoAI is looking for talented engineers, junior research scientists and research scientists to join their team. Elixir/Rust full stack engineer: https://invideo.notion.site/Elixir-Rust-full-stack-engineer-158316ee111a8044846be07038d3e481 Research scientist - generative AI: https://invideo.notion.site/Research-scientist-generative-AI-17c316ee111a8096bae4c7669b602dec If you want to apply for any of the ML or engineering roles, reach out to them at [email protected] ======================================================