​CERN and GÉANT

Helping CERN deal with the LHC data.

 

©CERN

 

Challenge

Approximately 600 million times per second, particles collide in each detector of the Large Hadron Collider (LHC), producing approximately one petabyte of data per second. None of today’s computing systems are capable of recording such rates. Hence sophisticated selection systems are used for a first fast electronic pre-selection, selecting one out of 10,000 events. Tens of thousands of processor cores then select 1% of the remaining events reducing the peak rate to a more manageable 80Gbps. The LHC data are then aggregated in the CERN Data Centre, where initial data reconstruction is performed, and a copy is archived to long-term tape storage. Another copy is sent to several large scale data centres around the world using a dedicated network called the LHC Optical Private Network (LHCOPN). Subsequently hundreds of thousands of computers from around the world come into action: harnessed in a distributed computing service, they form the Worldwide LHC Computing Grid (WLCG), which provides the resources to store, distribute, and process the LHC data.

  
Physicists must sift through the 30 petabytes or so of data produced annually to determine if the collisions have thrown up any interesting physics. They form a worldwide community of over 10,000 scientists using WLCG to have near real-time access to LHC data.

  
In 2012, the Data Centre at the Wigner Research Centre for Physics in Budapest began operating as an extension to the CERN Data Centre - both to cope with the increasing requirements for LHC computing and also to ensure full business continuity for the critical systems in case of a major problem on CERN’s site in Switzerland.

  

Solution

Research and education networks such as GÉANT and its European NREN (National Research and Education Network) partners provide the high-capacity bandwidth required by CERN to transport this data all over the globe. In particular, GÉANT and the NRENs provide 100Gbps IP access; 100Gbps LHCONE (LHC Open Networking Environment) access; access to the peering service; 100Gbps GÉANT Lambda connections between the CERN DC in Geneva and the extension in Budapest; dedicated lambdas as part of the LHCOPN; eduroam to provide roaming Wi-Fi.

  
These solutions are well-used:

  • The LHCOPN regularly sees peaks in excess of 60Gbps.
  • The 100Gbps IP access regularly sees peak traffic in excess of 20Gbps.
  • LHCONE regularly sees peak traffic in excess of 30Gbps.
  • In total, all the LHCONE traffic on the GÉANT pan-European network regularly sees peaks in excess of 60Gbps.

Results 

Without research and education networks such as GÉANT’s pan-European network and those of its European NREN partners providing fast, trusted and highly-reliable connectivity, these types of colossal science projects - which involve worldwide partners and have the potential to change our understanding of the world - could not take place.

   

In 2012, the Wigner Centre was one of the first beneficiaries of GÉANT’s terabit network, utilising multiple 100G links. At the time David Foster, Deputy Head of the CERN IT Department said, “The GÉANT network is fundamental to our data transfer needs, and we’re delighted that we will be continuing this successful relationship.”