Why Your RAG System Is Broken, and How to Fix It with Jason Liu - 709

Why Your RAG System Is Broken, and How to Fix It with Jason Liu - 709

1.836 Lượt nghe
Why Your RAG System Is Broken, and How to Fix It with Jason Liu - 709
Today, we're joined by Jason Liu, freelance AI consultant, advisor, and creator of the Instructor library to discuss all things retrieval-augmented generation (RAG). We dig into the tactical and strategic challenges companies face with their RAG system, the different signs Jason looks for to identify looming problems, the issues he most commonly encounters, and the steps he takes to diagnose these issues. We also cover the significance of building out robust test datasets, data-driven experimentation, evaluation tools, and metrics for different use cases. We also touched on fine-tuning strategies for RAG systems, the effectiveness of different chunking strategies, the use of collaboration tools like Braintrust, and how future models will change the game. Lastly, we cover Jason’s interest in teaching others how to capitalize on their own AI experience via his AI consulting course - https://maven.com/indie-consulting/ai-consultant-accelerator?promoCode=TWIML 🎧 / 🎥 Listen or watch the full episode on our page: https://twimlai.com/go/709. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Follow us on Twitter: https://twitter.com/twimlai Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/ Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/ 📖 CHAPTERS =============================== 00:00 - Introduction 2:38 - Tackling genAI challenges 6:44 - Evaluation loops, pipelines, and flywheels 9:50 - Building the data set 17:40 - Decision matrix for embedding implementation 23:25 - Evaluation tooling 26:29 - Compression rate 29:22 - Evaluation metrics 34:20 - Fine-tuning in RAG 36:34 - Long-context lengths 44:14 - Optimizations 48:51 - Multimodal 50:05 - Agentic programs 53:44 - AI consulting 🔗 LINKS & RESOURCES =============================== https://jxnl.co/ AI Consulting Course - https://maven.com/indie-consulting/ai-consultant-accelerator?promoCode=TWIML Systematically Improving RAG Applications Course - https://maven.com/applied-llms/rag-playbook?promoCode=TWIML 📸 Camera: https://amzn.to/3TQ3zsg 🎙️Microphone: https://amzn.to/3t5zXeV 🚦Lights: https://amzn.to/3TQlX49 🎛️ Audio Interface: https://amzn.to/3TVFAIq 🎚️ Stream Deck: https://amzn.to/3zzm7F5