Evolving MLOps Platforms for Generative AI and Agents with Abhijit Bose - 714

Today, we're joined by Abhijit Bose, head of enterprise AI and ML platforms at Capital One to discuss the evolution of the company’s approach and insights on Generative AI and platform best practices. In this episode, we dig into the company’s platform-centric approach to AI, and how they’ve been evolving their existing MLOps and data platforms to support the new challenges and opportunities presented by generative AI workloads and AI agents. We explore their use of cloud-based infrastructure—in this case on AWS—to provide a foundation upon which they then layer open-source and proprietary services and tools. We cover their use of Llama 3 and open-weight models, their approach to fine-tuning, their observability tooling for Gen AI applications, their use of inference optimization techniques like quantization, and more. Finally, Abhijit shares the future of agentic workflows in the enterprise, the application of OpenAI o1-style reasoning in models, and the new roles and skillsets required in the evolving GenAI landscape. 🎧 / 🎥 Listen or watch the full episode on our page: https://twimlai.com/go/714. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Follow us on Twitter: https://twitter.com/twimlai Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/ Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/ 📖 CHAPTERS =============================== 00:00 - Introduction 4:18 - Role of platforms at Capital One 7:50 - Platform users and interaction levels 12:18 - Relationship between traditional ML workloads and Gen AI 14:22 - Extending MLOps platforms to GenAI 18:22 - Model use and selection 20:35 - Fine-tuning 31:41 - Data annotation in traditional ML/AI vs GenAI 34:35 - Platform support for workflow automation 34:58 - Inference 38:32 - Optimization techniques 39:34 - GPU alternatives 43:00 - Agentic workflows 51: 28 - Future of GenAI at Capital One 🔗 LINKS & RESOURCES =============================== Learn more about AI at Capital One - https://www.capitalone.com/tech/ai/ AI career opportunities at Capital One - https://www.capitalonecareers.com/search-jobs/AI/234/1 Scaling AI: Platform best practices - https://venturebeat.com/ai/scaling-ai-platform-best-practices/ 📸 Camera: https://amzn.to/3TQ3zsg 🎙️Microphone: https://amzn.to/3t5zXeV 🚦Lights: https://amzn.to/3TQlX49 🎛️ Audio Interface: https://amzn.to/3TVFAIq 🎚️ Stream Deck: https://amzn.to/3zzm7F5