During SC18, ESnet staff will be working with other research organizations and vendors to support a series of demos at the forefront of networking, including the use of new hardware and software. These demos include:
SENSE: In collaboration with two universities and three national laboratories, ESnet will demonstrate its Software-defined Network for End-to-end Networked Science at Exascale research project.
FAUCET: ESnet is collaborating with the University of Waikato in New Zealand and the faucet Foundation on a very large, never-before-done deployment of the Faucet controller, which was created to bring the benefits of software-defined networking to a typical enterprise network and has been deployed in various settings.
Monon400: As part of the Network Research Exhibition (NRE) at SC18, ESnet is collaborating with Indiana University (IU), Ciena and Internet2 to power Monon400, the fastest network ever built for research and education.
“Applications want more instrumentation from the network, and so do network engineers,” said Monga. “As the era of ‘gut-feel networking’ passes to analytics-driven networking, more and more data about networks, including the constituent flows, will need to be being tracked and retrieved. With networks becoming an effective sensor, new methods are being proposed to manage the streaming telemetry.”
Upgrades include a programmable API, graphical interface and ability to make multi-point reservations
The Energy Sciences Network’s (ESnet’s) award-winning OSCARS (On-demand Secure Circuits and Reservation System) has just been upgraded. The tool, now called OSCARS 1.0, allows network engineers to make multi-point bandwidth reservations and features a new programmable API and graphical user interface.
OSCARS is a software service that creates dedicated bandwidth channels for scientists who need to move massive, time-critical data sets around the world. The previous iteration, called OSCARS version 0.6, was finalized and released in 2012. In 2013, it was recognized as one of R&D Magazine’s top 100 technologies.
“OSCARS 1.0 brings a lot of engineering improvements to the tool,” said ESnet Engineer Evangelos Chaniotakis, who led the upgrade. “We simplified the architecture so that engineers can connect more tools. We also made improvements to pathfinder so that users can choose between different flavors of path finding. Now instead of just looking for the shortest path, users can make requests like ‘find me the path with the least amount of devices,’ which ensures resiliency.”
According to Chaniotakis, the most visible change to the tool is the user interface, which now has an almost fully graphical view of the network and the connections on it. Before, users would see this information in a spreadsheet. He notes that with a few clicks of the mouse, OSCAR 1.0 users can get a pretty complex network setup with information about quality of service, network constrains, and scheduling, among other data. OSCARS 1.0 will also continue to support the MSI protocol for international collaborations.
“We’ve deployed OSCARS 1.0 at a few selected sites, and it is considerably faster. I was able set up hundreds of connections with this updated version in the same time it took me to do 10 connections with the previous version,” says John MacAuley, an ESnet Software Architect, who helped test OSCARS 1.0.
The tool also allows engineers to automatically provision jobs on the network, which is especially useful for long-haul, day-to-day tasks for large experiments like the Large Hadron Collider, as well as research projects that require complicated network topologies. Chaniotakis notes that this automatic provisioning is less burdensome for network engineers who used to manually set up these tasks. It will also become a crucial tool as ESnet continues to grow.
Using a 5,000-mile network loop operated by ESnet, researchers at the Department of Energy’s SLAC National Accelerator Laboratory (SLAC) and Zettar Inc. recently transferred 1 petabyte in 29 hours, with encryption and checksumming, beating last year’s record by 5 hours, an almost 15 percent improvement.
The project is aimed at achieving the high data transfer rates needed to accommodate the amount of data to be generated by the Linac Coherent Light Source II (LCLS II), which is expected to come online in 2020. The LCLS is the world’s first hard X-ray free-electron laser (XFEL) and its strobe-like pulses are just a few millionths of a billionth of a second long, and a billion times brighter than previous X-ray sources. LCLS II will provide a major jump in capability – moving from 120 pulses per second to 1 million pulses per second. Scientists use LCLS to take crisp pictures of atomic motions, watch chemical reactions unfold, probe the properties of materials and explore fundamental processes in living things.
The increased capability is expected to generate data transfers of multiple terabits per second– as the experimental results are sent from SLAC to Department of Energy’s (DOE) supercomputing facilities for analysis. As the DOE’s dedicated network user facility for scientific research, ESnet carries data between universities and DOE’s national laboratories and national user facilities along a multi-100 gigabits-per-second (Gbps) backbone network.
This screen shot from MyESnet shows the southern links of the network dark red, indicating data transfers exceeding 50 Gbps during the 5,000-mile transfer conducted by SLAC and Zettar Inc.
As ESnet’s new chief security officer, Adam Slagell will head the facility’s security team and generate a security strategy for the network. He will also join the organization’s senior leadership team and provide valuable input into the growing network’s strategic direction.
Slagell comes to ESnet from the National Center for Supercomputing Applications (NCSA) where he spent the last 15 years of his career and was promoted several times. After joining NCSA in 2003 as a security engineer, he was promoted to Senior Security Engineer in 2007, then to Senior Security Engineer and Chief Information Security Officer in 2012, Assistant Director of the Cybersecurity Directorate and Chief Information Security Officer in 2013, and Cybersecurity and Networking Division Director and Chief Information Security Officer in 2016.
During this time at NCSA, he also served as principal investigator and co-principal investigator on a number of grants. He was co-PI for the Bro Project in collaboration with researchers in UC Berkeley’s International Computer Science Institute and Berkeley Lab’s Computational Research Division and served as the first chair of the Bro Project’s leadership team. He was also security operations co-lead and the security officer for the XSEDE federation, a single virtual system that scientists can use to interactively share computing resources, data, and expertise; and developed the policies and procedures for the first HIPAA enclave on the University of Illinois at Urbana-Champaign campus.
“What brings me to ESnet is a chance to do something new and still be close to open scientific research,” said Slagell. “For most of my career I’ve been focused on data center security, but at ESnet we’re looking at security on a wide area network. We are providing high-bandwidth services to a whole bunch of customers around the world and we have a responsibility to protect those resources.”
Born and raised in Northern Illinois, Slagell received his masters degree in computer science from the University of Illinois at Urbana-Champaign in 2003, a masters and bachelors degree in mathematics from Northern Illinois University in 2000 and 1999, respectively.
“I was really ahead of the game in mathematics,” said Slagell. “I was taking college-level math classes in high school and in college I got into cryptography primarily though independent study. This eventually led me to computer security, which I loved because it allowed me to do pure math while staying close to computers, which is something I’ve always been interested in.”
Slagell will be working remotely from Illinois. He is an active power-lifter and biker. He rides his bike 10 miles every day to get to work, even in Illinois winter and summer. In his spare time, he is active in a charity that his wife co-founded called CU Able, which is a support network for families of individuals with disabilities.
Sponsored by the DOE’s Office of Advanced Scientific Computing Research, the event will bring together experts from national labs, industry, government, and academia to identify research challenges and opportunities in the development and deployment of quantum networks to support distributed quantum information science (QIS) activities. The workshop will cover a spectrum of quantum communications networks and subsystems, including quantum interconnects, quantum local area networks, quantum metropolitan networks, and quantum wide area networks.
“While we are busy designing and developing our next-generation network ESnet6, we need to look ahead to the emerging quantum technologies that will impact our science users to ensure that we have a deep understanding of the technological issues and are able to invest in the appropriate infrastructure when needed,” said Monga, who is on the organizing committee for the QNet workshop and will co-lead a discussion on Operations and Control for Quantum Networks with Dr. Ben Yoo from the University of California, Davis and Dr. Cees de Laat from the University of Amsterdam, an affiliate of Berkeley Lab and a member of the QuSoft Research Center on Quantum Software.
Summer student interns have a lot to offer Berkeley Lab: they are motivated, energetic and can bring fresh perspectives to old problems. They also represent the next generation of researchers and engineers. To capture the best talent, ESnet has built an internship program that operates as a student-to-staff pipeline, channeling not only talent but diversity into Berkeley Lab, said Patricia Giuntoli, area lead for Networking and Systems at ESnet.
“Diversity is important in order to have unique and different views. Sometimes solving a problem or getting to a new place can be challenging, and having people with different experiences and views can help create unique solutions,” said Giuntoli.
This summer, four students have been working at ESnet. Three of them—Hocine Mahtout, from the Bordeaux Graduate School of Engineering; Kunal Singh, from UC Berkeley; and Aaron Jia, who is also attending UC Berkeley—are working with ESnet Research Scientist Mariam Kiran on a project that employs machine learning and parallel computing to optimize network traffic and path allocation. In 2017, Kiran received an Early Career Award to support this work.
Jia and Singh are developing algorithms to predict network traffic and deep-learning methods to model network traffic. This information can be used to prepare for intense server traffic and help designate server workload. When they are done with the research, they will each write a paper about their findings and present their findings at SC18 in Dallas, TX.
Mahtout is also working on a separate project with Kiran. A few years ago, Kiran started working on iNDIRA, a software package that translates a user command written in English (not a programming language) into network commands. After seeing Kiran talk about iNDIRA in a presentation, Mahtout independently contacted Kiran to discuss ways they might work together. They later launched a second version of the project called EVIAN that incorporates machine learning to improve future interactions with users.
We did not intend on creating a new version of iNDIRA, but because Mahtout was so excited about working on it we now have a second version of iNDIRA coming out,” Kiran said. “As an intern you see the environment and the types of science happening here and making a difference to everyday life, and it’s so exciting that it makes you want to be part of it all.”
Meanwhile, the fourth student—John Christman, a senior from University of Nevada, Reno whose father, John Christman Sr., recently retired from ESnet after 29 years—is working with ESnet Network Engineer Nick Buraglio to improve ESnet security and traffic analysis software. Christman, who also worked at ESnet last summer doing network traffic analysis, is working with the monitoring group to improve ESnet’s networks by getting new servers and networks up and running and securing them from cybersecurity attacks.
“John has been tasked with working on the prototyping team for a next-generation management network design,” said Nick Buraglio, ESnet network engineer and Christman’s mentor. “John has excelled at both the tasks of leading pieces of a larger project as well as operating as a high performing team member. He has gained experience with completing deliverables that are critical pieces of a larger project.”
At the University of Nevada Reno, Christman is majoring in Information Systems, which is similar to computer science except geared towards use in enterprise. He plans to get his masters degree in this subject at UC Berkeley.
“I’ve learned an incredible amount while I’ve been here. From hearing presentations from world-class scientist to gaining hands-on experience, this summer has been filled with new insights on cutting-edge information,” he said. “My expectations have not only been met but exceeded. I look forward to coming in everyday, as each day brings something new and exciting. I’m truly blessed to be interning here this summer.”
One of ESnet’s goals with the summer student program is to give students the opportunity to gain a working knowledge of ESnet that might lead them to join the ESnet team once they graduate. Several ESnet employees, including Kiran, Sowmya Balasubramanian and Scott Richmond, began their careers at the lab as interns.
“My goal was to educate myself and gain practical experience in network operations so I could leverage that into a career in industry and the laboratory was a well-respected place that was accepting interns,” said Richmond, who is now a network engineer at ESnet. “After spending nearly a year at ESnet as a student assistant, the training and education I received from my mentors and colleagues in software, hardware and networking allowed me to transition to a full-time career as a Computer Systems Engineer with ESnet, working in the Operations and Deployment group.”
Some of this summer’s interns may just follow suit.
“This internship has gotten me thinking about what I want to do in the future,” said Jia. “I had never had formal work experience before, and this gave me a great idea of what it’s like.”
Last week, several ESnet staff members organized and participated in the second annual National Research Platform (NRP) workshop. Held August 6-7, 2018, at Montana State University in Bozeman, Montana, the workshop brought together representatives from more than 100 key institutions that are working to create a nationwide science data network.
Co-sponsored by a National Science Foundation (NSF) award to the California Institute for Telecommunications and Information Technology (Calit2) at the University of California, San Diego (UCSD), CENIC, ESnet and Internet2 and hosted by Montana State University, the workshop was designed to identify best strategies for enabling high-performance data transfer via interoperable Science DMZs and data transfer nodes at a national scale.
Several ESnet staff members helped organize and are participating in the second annual National Research Platform (NRP) workshop. Held August 6-7, 2018, at Montana State University in Bozeman, Montana, the workshop brings together representatives from more than 100 key institutions who are working to create a nationwide science data network.
Co-sponsored by a National Science Foundation (NSF) award to the California Institute for Telecommunications and Information Technology (Calit2) at the University of California, San Diego (UCSD), CENIC, ESnet and Internet2 and hosted by Montana State University, the workshop is designed to identify best strategies for enabling high-performance data transfer via interoperable science DMZs and data transfer nodes at a national scale. The Science DMZ is a secure portion of the network, built at or near the campus or laboratory’s local network perimeter that is tailored to the needs of high performance science applications, including high-volume bulk data transfer, remote experiment control, and data visualization.
The NRP is an outgrowth of the Pacific Research Platform (PRP), an NSF-funded cooperative agreement with UCSD and UC Berkeley formed in 2015 to improve the end-to-end high-speed networking data transfer capabilities in collaborative, big-data science. Today the PRP is a regional “superhighway” linking most of the research universities on the West Coast, as well as partner institutions in other parts of the U.S., via three advanced networks: CENIC’s California Research & Education Network, Pacific Wave and ESnet.
“The PRP is a regional DMZ that is built directly on the DMZ concept developed by DOE/ESnet, and that platform is what we are trying to scale up to the national level,” said Larry Smarr, principal investigator for the PRP, founding director of Calit2 and professor of computer science and engineering at the University of California, San Diego (UCSD). “The far goal is to have a national research platform based on the DMZ model.”
Smarr and Internet2’s Jim Bottum co-chaired the first NRP workshop, held in August 2017. For the 2018 workshop, Smarr is co-chairing with ESnet Director Inder Monga and Ana Hunsinger, vice president of Community Engagement of Internet2. The community-wide program committee is co-chaired by Marla Meehl, who manages the high-speed network for the University Corporation for Atmospheric Research, and Sherilyn Evans, COO of CENIC, and includes Kate Mace, Jen Leasure, Bottum, Maxine Brown and Wendy Huntoon.
“NRP is designed to bring together the networking practitioners, scientists and larger R&E community to figure out how to build scalable platforms that can make the best use of fast networks for science applications,” said Monga. “DOE and NSF have already invested a lot of effort in building scienceDMZs in national labs, supercomputing centers and universities. The PRP and NRP efforts are a way to build a sustainable, community-supported platform that will be transformational for science in the upcoming decade” The NRP also highlights the long-standing collaborations between NSF and DOE as well as Berkeley Lab and UCSD, he added.
In addition to Monga, Eli Dart, a network engineer in ESnet’s Science Engagement Group, is giving a keynote at the NRP workshop on “Modern Cyberinfrastructure: The Ladder to the Shoulders of Giants.” Mace, a science engagement engineer at ESnet, is participating in a panel session, “Scaling Across the NRP Ecosystem From Campus to Regional to National – What Support Is There?” and chairing another, “International-Scale Measurement Technologies/Techniques.”
With the NRP collaboration still in the prototyping and engineering optimization stage, this year’s workshop will focus on requirements from domain scientists and the networking architecture, policies, tools and security necessary to scale to hundreds of institutions participating in the NRP.
“To accelerate the rate of scientific discovery, researchers must get the data they need, where they need it and when they need it,” Smarr said. “This requires a high-performance data freeway system that connects data generators and users of that data. Through the NRP we are investigating what would be necessary to create a national DMZ that would link academic researchers at campuses all over the country.”