End to End LLMOps with Kubeflow - J. George, G. Prabhu, A. Nagar & A. Raimule, K. Durai

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon Europe in London from April 1 - 4, 2025. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io End to End LLMOps with Kubeflow - Johnu George, Gavrish Prabhu, Ajay Nagar & Aishwarya Raimule, Nutanix; Krishna Durai, Meta In the newer world of generative AI models, enterprises bet on integrating large language models into their various business use cases. Due to the complex infrastructure requirements of large language models, building scalable optimized end-to-end GenAI pipelines connecting data and compute is not easy compared to traditional machine learning models. Cluster admins need better visibility into infrastructure to ensure the best utilization of cluster resources, including expensive accelerators. In contrast, data scientists need a clean Pythonic interface without exposure to any underlying stack details. In this talk, we will cover how the Kubeflow Platform helps in LLMOps journey from training an LLM on the custom dataset to fine-tuning the pipeline for the best results and, finally, deployment of the trained models at scale. We will discuss an optimized Kubernetes native ML reference stack for your LLM needs that provides maximum infra utilization.