How the World’s Fastest Science Network Was Built

ESnetFireworks

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

Funded by DOE’s Office of Science and managed by the Lawrence Berkeley National Laboratory (Berkeley Lab), ESnet moves about 51  petabytes of scientific data every month. This is a 13-step guide about how ESnet has evolved over 30 years.

Step 1: When fusion energy scientists inherit a cast-off supercomputer, add 4 dialup modems so the people at the Princeton lab can log in. (1975)

Online3

Step 2: When landlines prove too unreliable, upgrade to satellites! Data screams through space. (1981)

18ogxd

Step 3: Whose network is best? High Energy Physics (HEPnet)? Fusion Physics (MFEnet)?  Why argue? Merge them into one-Energy Sciences Network (ESnet)-run by the Department of Energy!  Go ESnet! (1986)

ESnetListicle

Step 4: Make it even faster with DUAL Satellite links! We’re talking 56 kilobits per second! Except for the Princeton fusion scientists – they get 112 Kbps! (1987)

Satellite

Step 5:  Whoa, when an upgrade to 1.5 MEGAbits per second isn’t enough, add ATM (not the money machine, but Asynchronous Transfer Mode) to get more bang for your buck. (1995)

18qlbh

Step 6: Duty now for the future—roll out the very first IPv6 address to ensure there will be enough Internet addresses for decades to come. (2000)

18s8om

Step 7: Crank up the fastest links in the network to 10 GIGAbits per second—16 times faster than the old gear—a two-generation leap in network upgrades at one time. (2003)

18qlnc

Step 8: Work with other networks to develop really cool tools, like the perfSONAR toolkit for measuring and improving end-to-end network performance and OSCARS (On-Demand Secure Circuit and Advance Reservation), so you can reserve a high-speed, end-to-end connection to make sure your data is delivered on time. (2006)

18qn9e

Step 9: Why just rent fiber? Pick up your own dark fiber network at a bargain price for future expansion. In the meantime, boost your bandwidth to 100G for everyone. (2012)

18on55

Step 10: Here’s a cool idea, come up with a new network design so that scientists moving REALLY BIG DATASETS can safely avoid institutional firewalls, call it the Science DMZ, and get research moving faster at universities around the country. (2012)

18onw4

18oo6c

Step 11: We’re all in this science thing together, so let’s build faster ties to Europe. ESnet adds three 100G lines (and a backup 40G link) to connect researchers in the U.S. and Europe. (2014)

18qnu6

Step 12: 100G is fast, but it’s time to get ready for 400G. To pave the way, ESnet installs a production 400G network between facilities in Berkeley and Oakland, Calif., and even provides a 400G testbed so network engineers can get up to speed on the technology. (2015)

18oogv

Step 13: Celebrate 30 years as a research and education network leader, but keep looking forward to the next level. (2016)

ESnetFireworks

Aug. 21 Deadline to Submit Papers for SCinet’s INDIS Workshop at SC16


Sunday, Aug. 21, is the deadline for submitting papers for the 3rd International Workshop on Innovating the Network for Data Intensive Science (INDIS), which will be held Sunday, Nov. 13, in conjunction with SC16 in Salt Lake City, Utah.

The workshop invites papers that propose new and novel techniques regarding capacity and functionality of networks, its control and its architecture to be demonstrated at current and future supercomputing conferences. We invite full original papers to be submitted via the FGCS submission system (for more detailed information visit the submit page.

INDIS also serves as a platform for participants in SCinet to present experimental papers on their latest designs and solutions. SCinet is the high-speed network engine of the SC conference.

For complete information, including links to papers from the 2015 INDIS workshop, go to: https://scinet.supercomputing.org/workshop/

For 10 Years, ESnet has Helped Develop, Deploy perfSONAR


perfSONARv2Since it was first deployed as a prototype in December 2005, the perfSONAR toolkit has provided the research and education networking community with tools for end-to-end monitoring and troubleshooting of multi-domain network performance. And over the years, this ability to diagnose network problems has become increasingly important as research is increasingly collaborative and dependent on sharing large data sets.

Currently a joint effort between ESnet, Internet2, Indiana University and GEANT, the pan-European research network, perfSONAR is now deployed at more than 1,700 public sites around the world, as well as at many private sites. About 40 percent of the public sites are at educational institutions. Among the major users is the Large Hadron Collider collaboration with users at hundreds of institutions.

“Although it started out with just one of two metrics of network performance, it’s now a menagerie of tools in one easy-to-install package with not a lot of steps required to install and configure,” said Jason Zurawski of ESnet’s Science Engagement Team. “At its core, perfSONAR is software to fix your network, whether it’s broken or improperly tuned. We find that many new users are tired of poor performance and want to fix it, or they are looking to upgrade their network and want to benchmark the current system to document improvement.”

Read the full story.

ESnet_30year_r3_Final(Editor’s note: This is the first of a series highlighting ESnet’s contributions to the global networking community as ESnet marks its 30th anniversary.)

 

ESnet, NSRC Team Up to Produce Videos for Boosting Network Performance


perfSONAR screen

The Department of Energy’s ESnet and the Network Startup Resource Center (NSRC) at the University of Oregon are teaming up to create an extensive video training library to help organizations improve the performance of their networks by deploying the perfSONAR network measurement tools and the Science DMZ network architecture.

One of the first institutions to benefit from the project will be the University of Guam. NSRC staff are flying to the U.S. territory in the Pacific on July 25 to help the university improve its campus network and set up a high-speed link to the University of Hawaii. Prior to the weeklong workshop and perfSONAR installation, NSRC staff will have the six campus network engineers familiarize themselves with the toolkit via the video library.

ESnet is one of the original developers of the perfSONAR toolkit, which has provided the research and education (R&E) networking community with tools for end-to-end monitoring and troubleshooting of multi-domain network performance. Other current partners in the perfSONAR project are Internet2, Indiana University and GÉANT, the pan-European research network. The perfSONAR collaboration over the past 11 years has also included Fermilab, SLAC, Georgia Tech, the University of Delaware, the ATLAS Great Lakes Tier 2 team at the University of Michigan and RNP in Brazil.

“We’ve developed a good working relationship with NSRC over the past several years, and we started talking about this video project in early 2015 as a way to efficiently transfer ESnet’s expertise to anyone interested in network performance, from DOE scientists to other research organizations,” said ESnet’s Eli Dart, who developed the Science DMZ architecture and will appear in a series of related training videos. “They do a lot of good work in all corners of the world, helping research organizations in emerging economies around the world get up to speed fast and help them bootstrap the development of science networks where it was not possible before.”

Read the full story.

Inder Monga Named Director of ESnet


Inder_Monga
Inder Monga is the new ESnet director.

Indermohan “Inder” Singh Monga, an internationally recognized expert in advanced networking research, is the new executive director of the Department of Energy’s Energy Sciences Network, better known as ESnet. He will also assume the role of director of the Scientific Networking Division at Lawrence Berkeley National Laboratory, which manages ESnet.

Monga, who joined ESnet in September 2009, is only the fifth person to lead ESnet since it was created 30 years ago. When Greg Bell announced he was stepping down as ESnet director in February 2016, Monga was named interim director. Since joining the organization, Monga has served as a software engineer, chief technology officer, group lead of the Tools Team and deputy of technology for the Scientific Networking Division. He provides research and technology direction, actively leads research projects and championed building a focused software engineering effort within ESnet. He is also a frequent invited/keynote speaker at industry and research and education (R&E) networking conferences.

“ESnet was very fortunate to have Inder join the organization in 2009 and I’m very pleased that he will now lead ESnet into the next generation of scientific networking,” said Kathy Yelick, Associate Lab Director for Computing Sciences at Berkeley Lab, in announcing Monga’s appointment. “As science has become increasingly science-driven, networking plays a critical role in giving researchers access to critical data, as well as supporting global collaborations.”

Read more at: http://es.net/news-and-publications/esnet-news/2016/inder-monga-named-director-of-esnet-berkeley-lab-s-scientific-networking-division/

Interactive Timeline Showcases ESnet’s 30 Years of Networking Leadership


ESnet_30year_r3_FinalIn observance of its 30 years of supporting Department of Energy sciences, ESnet is kicking off six months of activities highlighting the network’s accomplishments and contributions to the global networking community, starting with an interactive timeline highlighting ESnet’s networking innovations and milestones, and including extensive documentation of the network as it evolved.

In 1986, DOE merged the Magnetic Fusion Energy Network (MFEnet) with the High Energy Physics Network (HEPnet) to create the Energy Sciences Network, better known as ESnet.  But ESnet’s roots reach back to the mid-1970s when four dial-up modems connected the Princeton Plasma Physics Lab to the National Magnetic Fusion Energy Computer Center, which is today known as NERSC, DOE’s National Energy Research Scientific Computing Center.

In the coming months, watch for more posts marking ESnet’s 30th.

View the timeline.

ESnet’s John Christman Retiring after 29+ Years at LBNL


JC_Univac
John Christman with his Navy-issue UNIVAC computer.

When John Christman enlisted in the U.S. Navy the day after his 18th birthday, he was hoping to work as either a welder or machinist. But he scored too high on the skills assessment and instead ended up studying advanced electronics. It was one of those twists of fate that set his whole career in motion. On June 29, Christman will leave the lab after 29+ years, the last 10 as a network engineer with ESnet.

He started out with two years of electronics school in the Navy, going to class eight hours a day and learning about vacuum tubes, transistors and programming. And while the technology changed, the nature of Christman’s work  has remained constant – understanding and maintaining systems to ensure that users were able to reliably get the information they needed.

“Every year you learn something new – that’s what I love about the lab,” he said. “You’re always at the cutting edge, nothing gets stale.”

Read the full story, including John’s prize winning Whiskey-Peppercorn Steak recipe.

JC_Edison2
In his last week with ESnet, John Christman posed with NERSC’s Edison supercomputer.

ESnet, NSF Cybersecurity Center of Excellence Organize Working Group on Open Science Threats


Managing the security risks to scientific instruments, data and cyberinfrastructure is a priority for creating a trustworthy environment for science. Deep experience in understanding cybersecurity and the science being supported is needed. To achieve this,  ESnet and the NSF Cybersecurity Center of Excellence are collaborating with research and education community leaders to develop a threat profile for open science to formally capture and benchmark this expertise, allowing other organizations to apply these best practices more broadly.

Organized by Sean Peisert of Berkeley and Michael Dopheide from ESnet and Von Welch and Susan Sons from the NSF Cybersecurity Center of Excellence, a working group of nine scientists and cybersecurity leaders from across the country has been formed to tackle developing the threat profile: Ilkay Altintas (San Diego Supercomputer Center), RuthAnne Bevier (Caltech), James Cuff (Harvard), Rich LeDuc (Northwestern), Pascal Meunier (HUBzero), Reagan Moore (iRods), Stephen Schwab (USC Information Sciences Institute) and Karen Stocks (Scripps Institution of Oceanography).

“Several government and academic organizations involved in cybersecurity policy have built a solid foundation for risk management, but it still takes expert judgment to assess risks for the assets found in the open science community,” said Peisert, who in addition to his role at LBNL is chief scientist for cybersecurity at CENIC . “The goal of this effort is to provide tailored guidance to the science community on the threats to science assets and the consequences of those threats to the science mission. This information will provide a basic knowledge framework to expedite managing those threats for the wide portfolio of open science projects.”

Read the full story.

ESnet Internship Leads to Book Chapter on Energy Efficiency for Datacenters, Networks


Aksanli-photo1
Baris Aksanli

Just as research and education networks have expanded the capabilities of scientists, a 2011 internship at ESnet led Baris Aksanli, then a Ph.D. student in computer science and engineering from the University of California San Diego, to broaden the scope of his research in tapping renewable energy to reduce the carbon footprint of data centers and networks. The authors found that centers can increase their use of renewable energy while also reducing costs.

The result is a chapter in a new book on Computational Sustainability published on May 30, 2016, by Springer. Aksanli, along with Jagannathan Venkatesh and Tajana Simunic Rosing of UC San Diego and ESnet’s Inder Monga collaborated on the chapter Comp_Sust_coveron “Renewable Energy Prediction for Improved Utilization and Efficiency in Datacenters and Backbone Networks.”

Aksanli said his personal interest in energy efficiency led to studying how a single datacenter could become more energy efficient, but after working at Berkeley Lab and ESnet, the natural extension was to look at a group of networked datacenters, such as Department of Energy computing centers connected via ESnet.

Read the full story.

Just starting out? Apply for the SC16 Early Career Program


Applications are being accepted through Saturday, July 31 for the SC16 Early Career SC16Program for Professionals. The Early Career Program is designed to equip participants for significant contributions in their new careers by building skills related to obtaining funding, identifying publishing venues, establishing long-term mentor relationships, and effectively managing their time. The program is intended for people in their first five years of a permanent position (such assistant professors, researchers and technical staff members.)

SC16 will be held Nov. 13-18 in Salt Lake City, Utah.

Sessions will be held in a workshop format on Monday, Nov.  14 before SC16 starts to allow participants to enjoy the full technical program. The program includes formal presentations, time for peer mentoring, a speed mentoring session to meet 4-5 potential long-term senior mentors and an organized lunch for informal networking.

Learn more at: http://sc16.supercomputing.org/submitters/early-career-program/

Apply at: https://submissions.supercomputing.org/