Leonardo Pre-exascale System: Configuration and Hosting Site

Mirko Cestari, CINECA: CINECA will host one of the three recently procured precursor of exascale systems. Leonardo supercomputing system [1] will be installed on the newly built data center located in the Bologna Big Data Technopole (henceforth denoted as Technopole). The Emilia-Romagna Region supported CINECA’s candidacy to act as hosting entity for the pre-exascale system by providing part of an old tobacco manufacture buildings that eventually became the new data center. This action was deemed consistent with the plan to develop a big data thematic Technopole that would also include the hosting of INFN main data center, the Tier-1 of CNAF in Bologna. Emilia Romagna Region and ministry of university and research had already established a collaboration in order to promote the Technopole project to a national and international level. This collaboration was successful in obtaining the decision from the Council of the international intergovernmental body ECMWF to relocate its own data center in Bologna Technopole. Therefore, by virtue of hosting ECMWF, CINECA and INFN data centers, Bologna Technopole has risen to become one of the main European hubs for computing and data processing. CINECA and INFN new data center will follow a two-stage evolution plan. In the first stage (2021-2025) the data center will feature: 10 MW of IT load, 1240 m2 of computing room floor space, 900 m2 of ancillary space, a direct liquid cooling capacity of 8MW, a chilled water (18° -23° ) cooling capacity of 6+2 MW and power capacity of 3+1 MW (no-break) and 9+3 MW (short-break). The second stage (2025-2030) will see an increase to 20 MW IT load available and additional 2600 m2 computing room space floor available, while having mechanical and electrical infrastructure able to comply with two different expansion strategies, stage 2a: “Liquid Cooling Expansion” (16 MW direct liquid cooled + 4 MW air Cooled), or stage 2b: “Air Cooling Expansion” (8 MW direct liquid cooled + 12 MW air cooled). In designing the data center, particular care was devoted to containing the PUE, that for Leonardo is estimated below 1.1. Simulation of the PUE was based on loads and losses calculated for all systems (IT, mechanical and electrical) and on Bologna’s historical external conditions, following a strategy compliant with Level 3 Green Grid/ASHRAE. CINECA covered all the technical aspects of the procurement procedure and the design of the supercomputing system architecture, significantly promoting the competition between the candidates which resulted in final offers outbidding the tender minimum requirements. In fact, the result of the procurement procedure is a system capable of nearly 250 PFlops and equipped with over 100 petabytes of storage capacity. It will be based on Atos BullSequana XH2000 technology, with over 13,000 GPUs based on NVIDIA Ampere architecture and NVIDIA Mellanox HDR InfiniBand. The system will provide 10-20 times the computing power of the current CINECA flagship system Marconi-100.