The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis

If Charles Dickens was alive in 2024, A Tale of Two Cities might be the divide between the “GPU poor” and the “GPU rich”. We mentioned these terms in some of our previous episodes; they were originally coined by Dylan Patel of SemiAnalysis in his “Gemini Eats the World” post, put on blast by Sam Altman. SemiAnalysis are one of the most in depth research and consulting firms in the semis world, and have a unique insight into the design, production, and supply chain of GPUs based on their ground presence in Asia. In this episode we break down the State of Silicon: when are more GPUs coming? Are there real GPU alternatives on the way? Should Microsoft buy AMD chips just to scare Jensen? Is there a “GPU poor is beautiful” manifesto? Full show notes: https://www.latent.space/p/semianalysis 0:00 - Introductions 4:31 - Importance of infrastructure and hardware for AI progress 10:53 - GPU-rich vs GPU-poor companies and competing in AI 14:22 - Optimizing hardware and software for AI workloads 17:00 - Metrics for model training vs inference optimization 21:38 - Networking challenges for distributed AI training 23:19 - Google’s partnership with Broadcom for TPU networking 28:04 - What GPU-poor companies/researchers should focus on 34:47 - Innovation in AI beyond just model scale 38:03 - AI hardware startups and challenges they face 46:15 - Manufacturing constraints for cutting edge hardware 50:36 - Apple and AI 54:18 - AI safety considerations with scaling AI capabilities 57:37 - Complexity of rebuilding semiconductor supply chain 1:00:08 - Recommended readings to understand this space 1:04:31 - Dylan’s process for writing viral blog posts 1:07:27 - Dylan’s “magic genie” question