The CONNECT interview: Joe Metzger, ESNET


Joe Metzger, Senior Network Engineer at Lawrence Berkeley National Laboratory and member of the ESnet network engineering team, has spent the past year working at CERN. CONNECT caught up with him to hear about his experiences, and talk science support.


Joe, please tell us a little about ESnet

In the late 1970s and early 1980s the U.S. Department of Energy (DOE) funded HEPnet for high energy physics research as well as MFEnet for the magnetic fusion community. At one point it became clear that not every DOE science program could have their own network and ESnet was founded in 1986 with the mission to support whatever science DOE funds. Be that high energy physics (HEP), fusion, genomics, climate, bioscience or other physical sciences. ESnet supports the science which is carried out by a group of DOE-funded labs and their collaborators.


Can you describe your role in ESnet?

Well, I have only been in ESnet for 20 years! I missed the entire first decade. Before I joined ESnet, I was at an ESnet customer site. My first team in ESnet was the Information Services Group. At the time, we were supporting mail, gopher, usenet news and were running an ISDN-based video-conferencing service. About 16 years ago, I switched to the Network Engineering Group and have been a part of network engineering ever since.


Initially I was focused on typical inward facing network engineering efforts, including designing, deploying and operating the ESnet network. But over time I started to spend more time looking at the big picture. Undetected soft errors that didn’t impact local connectivity but prevented wide area end-to-end high performance were very common. So I got involved in the perfSONAR effort to help build a framework and community to find and fix those errors. High Energy Physics (HEP) is responsible for a large percentage of ESnet’s traffic, so I started trying to figure out how ESnet could better meet their needs. Early on I was focused on how we should support CERN’s Large Hadron Collider (LHC) within the ESnet network. But that quickly evolved into participating in the effort to define the LHCOPN (LHC Optical Private Network) and LHCONE (LHC Open Network Environment) frameworks used by networks around the world to support the LHC community.


How do you see big project user support developing over the next three to five years? What would you say the GÉANT community needs to do?

Really, I think that supporting big science is actually the easy task. Big science normally has excellent facilities, long-term planning, sufficient resources and inhouse experts. We find it straightforward to work with them to figure out where they are going and how we can support them best. It is much trickier when it comes to small and mid-size projects. Here we see large data-sets being produced, but in many cases there is a lack of certain skills, tools and a reduced network expert base available, so they are not effectively using the network. Over the years, ESnet has approached this through our science engagement efforts, including the Fasterdata.es.net website and Operating Innovative Networks workshops. This train-the trainers approach, with the objective to push the networking knowledge and skills needed to support modern science workflows deeper into our community, is important for all of us. But here is the crux: what happens to ESnet or say GÉANT, if all our engagement efforts actually translate into traffic growth – are we ready for this?


I realised over the years that one of the bigger challenges for GÉANT is how removed you are from the scientists and end users. You have to work through the layers of NRENs, regional (national) connectors and campus networks to communicate with endusers. But I have also seen significant improvements, the most striking of which is around performance. We now get great response times from all the layers across Europe when we discover a performance problem. It is good to see that perfSONAR is increasingly standard across Europe. It makes a real difference in how we can jointly tackle performance issues end-to-end.


What would you really like to say to scientists so that we could support them better?

I think we are in the middle of some interesting times for networking support of science. Every day, more science teams are adopting automated workflow engines, distributed or cloud computing models, and experimental designs that depend on resources at multiple locations. At the same time, our regional, national, and international networks are growing more complex and more dynamic. But I don’t think anyone fully understands the end-to-end paths. There are going to be challenges when changes by one party take the others by surprise.


​So scientists, when the network providers describe their future plans, please listen, and don’t hesitate to keep asking questions until you understand how it is going impact you. Also, please talk to your network providers early when you are thinking about making significant changes to your workflows or designing new experiments that will require significant network resources. Finally please speak up if the networks are not meeting your needs. We can’t build the right network infrastructure and services if end users don’t tell us what they need.


You have been in Geneva now for almost a year - what was that like?

I came to Geneva to work for a year based at CERN. My wife Laura and I moved from Ames, Iowa, which is a small university town where half of the population disappears when the students leave campus in the summer, to the city centre of Geneva. It makes for quite a difference! From Ames I can drive 250km in almost any direction and mostly what I will see is flat fields of corn and soybeans. In Geneva there are different mountain ranges, countries and cultures, all less than two hours away. But it is the little things that are most striking. Public transport is great. Grocery stores are closed in the evenings and on Sundays, but interestingly the barber shop is open past 9:30 on Saturday nights. What I like about Geneva is that it is so varied. There are so many festivals – sometimes when I open the window, I can hear music playing from a festival by the lake, but other times it is so quiet and I can actually hear birds singing. That surprised me.