Orchestrating Quasi-Real Time Data Processing in the Computing Farm of the ATLAS Experi... G. Avolio

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon Europe in London from April 1 - 4, 2025. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io Orchestrating Quasi-Real Time Data Processing in the Computing Farm of the ATLAS Experiment at CERN - Giuseppe Avolio, CERN What has Kubernetes got to do with a High Energy Physics experiment collecting one million physics events per second at a data rate of 5 TB/s? That is what we would like to show you! The ATLAS experiment at CERN filters one million complex collision signatures per second provided by the Large Hadron Collider in quasi real-time, using a mixture of custom electronics and a large computing farm (the Event Filter – EF – farm) consisting of up to 5000 commodity servers. In this talk, we will tell you how we are going to exploit Kubernetes to orchestrate the ATLAS EF computing farm. In particular, we will focus on the strategy and optimizations we put in place in order to start more than 25000 PODs over more than 2500 worker nodes in about 50 seconds. We will also show the impact of the Kubernetes Scheduler and Controller Manager QPS values on POD start and stop throughputs and we will report about how custom scheduler profiles allow us to schedule PODs at an average rate of about 500 Hz.