ESnet, Network Startup Resource Center Combine Expertise to Spread the Word
For members of the established research and education (R&E) networking community, attending conferences or sitting in on workshop sessions is the normal way to learn about the latest equipment, architecture, tools and technologies.
But for network engineers striving to establish basic R&E infrastructure where bandwidth and other resources are scarce, the University of Oregon’s Network Startup Resource Center (NSRC) is often the primary information conduit. NSRC staff travel to emerging nations in Africa, Asia-Pacific, Middle East and South America where they hold intensive hands-on training courses combined with direct engineering assistance to bring institutions up to speed.
And for the second time in a year, ESnet and the NSRC have produced and released a library of short explanatory videos to help network engineers around the world gain basic knowledge, set up basic systems and drill down into areas of specific interest. In December, 15 videos detailing the Science DMZ network architecture were posted, covering the background and structure, specific designs, and techniques and technology.
“The goal is to make the information more accessible to networking staff, in the U.S. and particularly in emerging economic areas where institutions are trying to bootstrap a research network,” said ESnet Network Engineer Eli Dart, who developed the Science DMZ concept with Brent Draney of the National Energy Research Scientific Computing Center (NERSC). Both ESnet and NERSC are DOE Office of Science User Facilities managed by Lawrence Berkeley National Laboratory.
KINBER, the Keystone Initiative for Network Based Education and Research in Pennsylvania, and ESnet, the Department of Energy’s Energy Sciences Network, announced today (Nov. 18) the establishment of a new peering connection between KINBER’s PennREN network and the ESnet national network via Internet2’s AL2S service.
The new 10 gigabit-per-second (Gbps) based peering connection will allow for greater capacity, speed and enhanced research capabilities for several PennREN connected higher education institutions across Pennsylvania, including Swarthmore College, Drexel University, Pennsylvania State University, Cheyney University of Pennsylvania, Carnegie Mellon University, University of Pittsburgh, and Lehigh University.
Research applications that will benefit from this new enhanced connection include those in areas such as exascale computing; machine learning algorithms for matching theories, simulations and observations in cosmology; chemical imaging and other advanced Department of Energy-funded basic research in physical science.
ESnet is collaborating with Stanford University and SLAC to present twopresentations and demonstrations of high speed data transfer as part of the SC16 conference. If you’re in Salt Lake City, drop by and watch the data fly.Please also forward this invitation to others that may be interested.
3-4 p.m. Tuesday, Nov. 15, in the Stanford/SLAC booth #2101 we will be presenting three talks on:
The science (Linear Coherent Light Source-II) needs to transfer regularly massive amounts of data (1 Tbit/sec by 2024) from the experiment in Menlo Park (SLAC) to an Exascale computer in Berkeley (NERSC)
How ESnet is enabling distributed data intensive science and hence LCLS-II
A demonstration of transferring Lots of Small Files data motivated by LCLS’s need for semi real-time transfer and access to the data acquisition system’s output.
11 a.m.-12 noon Wednesday, Nov. 16, in the Department of Energy booth #1030 we will be demonstrating high-speed file-to-file transfers.
This will include a cost effective, energy and space efficient, high availability forward looking software and data transfer system reference design for high speed file transfer. We will demonstrate:
Over the LAN file to file data transfer of 10 x100GiB files over 2 x 100Gbps links.
Over the wide area (122ms link) with a 70Gbps ESnet OSCARS shared circuit including Lots of Small Files (1 million x 1MByte files) transfer with and without TLS encryption
Each year, more than 10,000 professionals in high performance computing, networking, data storage and analysis convene from around the world at the SC conference. For six days, the latest innovations, technologies and ideas will be shared during SC16 in Salt Lake City, with significant support from ESnet.
ESnet’s expertise in networking will be shared in workshop sessions, talks, demonstrations and overall support for the conference. In particular, This year, ESnet is helping provision 600 Gbps of bandwidth (6 x 100G) between ESnet and the SC16 conference, which is more than the network has ever brought up to SCinet in the past.
Over the past several months, ESnet and the NSF Cybersecurity Center of Excellence collaborated with research and education community leaders to develop a risk profile for open science to formally capture and benchmark this expertise, allowing other organizations to apply these best practices more broadly.
Today, the group is releasing its draft Open Science Cyber Risk Profile (OSCRP) and inviting comment from the research community. The OSCRP is designed to help principal investigators and their supporting information technology professionals assess cybersecurity risks related to open science projects. The draft document, along with information on how to comment, can be found at http://trustedci.github.io/OSCRP/.
Managing the security risks to scientific instruments, data and cyberinfrastructure is a priority for creating a trustworthy environment for science. Assessing, understanding and managing concerns of open science to explicitly capture risks to its integrity and availability, and sometimes also privacy issues, involves making judgments on the likelihood and consequences of risks. Deep experience in understanding cybersecurity and the science being supported is needed to achieve these goals.
The group invites comments on the document prior to final publication in early 2017. Longer-term, the document is intended to be a living, community document, being updated as open science computing evolves, and also as new approaches to security arise.
The Center for Trustworthy Scientific Cyberinfrastructure (CTSC) is funded as the National Science Foundation’s Cybersecurity Center of Excellence. The mission of CTSC is to improve the cybersecurity of NSF science and engineering projects, allowing those projects to focus on their science endeavors. This mission is accomplished through one-on-one engagements with projects to address their specific challenges; education, outreach, and training to raise the state of security practice across the scientific enterprise; and leadership on bringing the best and most relevant cybersecurity research to bear on the NSF cyberinfrastructure research community.
The Energy Sciences Network (ESnet) is an international, high-performance, unclassified network built to support scientific research. Funded by the U.S. Department of Energy’s Office of Science (SC) and managed by Lawrence Berkeley National Laboratory, ESnet provides services to more than 40 DOE research sites, including the entire National Laboratory system, its supercomputing facilities, and its major scientific instruments. ESnet also connects to over 140 research and commercial networks, permitting DOE-funded scientists to collaborate productively with partners around the world.
Funded through a grant from the National Science Foundation (NSF) and directly from ESnet, the program funds eight early to mid-career women in the research and education (R&E) network community to participate in the 2016 setup, build out and live operation of SCinet, the Supercomputing Conference’s (SC) ultra high performance network. SCinet supports large-scale computing demonstrations at SC, the premier international conference on high performance computing, networking, data storage and data analysis and is attended by over 10,000 of the leading minds in these fields.
The SC16 WINS program kicked off this week as the selected participants from across the U.S., headed to Salt Lake City, the site of the 2016 conference to begin laying the groundwork for SCinet inside the Salt Palace Convention Center. The WINS participants join over 250 volunteers that make up the SCinet engineering team and will work side by side with the team and their mentors to put the network into full production service when the conference begins on November 12. The women will return to Salt Lake City a week before the conference to complete the installation of the network.
“We are estimating that SCinet will be outfitted with a massive 3.5 Terabits per second (Tbps) of bandwidth for the conference and will be built from the ground up with leading edge network equipment and services (even pre-commercial in some instances) and will be considered the fastest network in the world during its operation,” said Corby Schmitz, SC16 SCinet Chair.
The WINS participants will support a wide range of technical areas that comprise SCinet’s incredible operation, including wide area networking, network security, wireless networking, routing, network architecture and other specialties.
“While demand for jobs in IT continues to increase, the number of women joining the IT workforce has been on the decline for many years,” said Marla Meehl, Network Director from UCAR and co-PI of the NSF grant. “WINS aims to help close this gap and help to build and diversify the IT workforce giving women professionals a truly unique opportunity to gain hands-on expertise in a variety of networking roles while also developing mentoring relationships with recognized technical leaders.”
“Not only is WINS providing hands-on engineering training to the participants but also the opportunity to present their experiences with the broader networking community throughout the year. This experience helps to expand important leadership and presentations skills and grow their professional connections with peers and executives alike,” said Wendy Huntoon, president and CEO of KINBER and co-PI of the NSF grant.
The program also represents a unique cross-agency collaboration between the NSF and DOE. Both agencies recognize that the pursuit of knowledge and science discovery that these funding organizations support depends on bringing the best ideas from people of various backgrounds to the table.
“Bringing together diverse voices and perspectives to any team in any field has been proven to lead to more creative solutions to achieve a common goal,” says Lauren Rotman, Science Engagement Group Lead, ESnet. “It is vital to our future that we bring every expert voice, every new idea to bear if our community is to tackle some of our society’s grandest challenges from understanding climate change to revolutionizing cancer treatment.”
2016 WINS Participants are:
Denise Grayson, Sandia National Labs (Network Security Team), DOE-funded
Julia Locke, Los Alamos National Lab (Fiber and Edge Network Teams), DOE-funded
Angie Asmus, Colorado State (Edge Network Team), NSF-funded
Kali McLennan, University of Oklahoma (WAN Transport Team), NSF-funded
Amber Rasche, North Dakota State University (Communications Team), NSF-funded
Jessica Shaffer, Georgia Institute of Tech (Routing Team), NSF-funded
Julia Staats, CENIC (DevOps Team), NSF-funded
Indira Kassymkhanova, Lawrence Berkeley National Lab (DevOps and Routing Teams), DOE-funded
The WINS Supporting Organizations: The University Corporation for Atmospheric Research (UCAR) http://www2.ucar.edu/
The Keystone Initiative for Network Based Education and Research (KINBER) http:www.kinber.org
This month, the Department of Energy’s ESnet co-located its bi-annual Site Coordinators Committee (ESCC) meeting with The Quilt’s Fall Meeting, as well as the National Science Foundation’s (NSF’s) Campus CyberInfrastructure and Cybersecurity Innovation for Cyberinfrastructure P.I. Workshops at the Marriott Downtown in Philadelphia, Penn. Holding all four meetings in one location allowed engineers and researchers to have fruitful discussions that highlight areas of science and related cyberinfrastructure that are important to the combined communities.
The co-located agenda included panel discussions on topics such as:
“National Cybersecurity,” facilitated by Berkeley Lab’s Sean Peisert. ESnet’s Michael Sinatra was a panelist.
“Science Engagement – Bridging the Gaps,” facilitated by ESnet’s Lauren Rotman. Berkeley Lab’s Craig Tull was a panelist.
“Climate Science Infrastructure Support,” facilitated by Colorado State University’s Christos Papadopoulos. Travis O’Brian from Berkeley Lab’s Earth Sciences Area was a panelist.
A unique collaboration between a U.S. telecommunications equipment provider and a Department of Energy (DOE) Office of Science national laboratory has helped dramatically improve design cycle times for future high-speed optical networking components.
Computer scientists and mathematicians from Lawrence Berkeley National Laboratory’s (Berkeley Lab) Computing Sciences organization worked with engineers at Ciena, a leading networking company, to speed up the process by which Ciena validates the design of its ASIC (application-specific integrated circuit) chips. The collaboration grew out of the existing relationship between Ciena, a pioneer in high-bandwidth optical transport technology, and the DOE’s Energy Sciences Network (ESnet), which uses Ciena products to support its high-speed network.
“Modern science requires fast global networks like ESnet, and almost all DOE network traffic happens to run over optical gear from Ciena,” said ESnet Director Inder Monga. “ESnet doesn’t transport quite as much traffic as, say, Google, but we have the unique challenge of coping with massive single data streams from detectors and instruments like the Large Hadron Collider, the world’s largest and most powerful particle accelerator. This means we need to exploit leading-edge networking components, and we’re often early adopters of those components.”
Update to underlying internet protocol promises to improve data transmission performance
With Google’s Sept. 16 release of the new TCP congestion control algorithm, BBR (Bottleneck Bandwidth and Round Trip Time/RTT), ESnet staff are gearing up to test this promising development.
According to early reports, Google engineers reported that the new algorithm “significantly increased throughput and reduced latency.” Google cited an increase in the speed of data transfers from three megabits per second (Mbps) to 9,150 Mbps on a test link with 1% packet loss and a latency of 100 milliseconds. Early testing by ESnet engineers has shown improvements from 10x to 100x on some international paths, but no improvements on other paths.
“I’m eager to work with our Google collaborators to understand the details of how BBR works and how it may benefit data-intensive science applications in the future,” said Brian Tierney, head of ESnet’s Advanced Network Technologies Group and co-creator of ESnet’s Fasterdata website. “This is potentially a great step forward for TCP, and we’re eagerly testing BBR TCP. We’ll post results as we get them.”
More technical details will be revealed when a description of BBR appears in the September-October 2016 issue of ACM Queue. Among the developers of BBR is Van Jacobson, who was a primary contributor to the original TCP in the 1980s when he worked at Lawrence Berkeley National Lab, where he led the Network Research Group.
Tierney said ESnet has begun testing BBR using the 2,000 instances of the perfSONAR network performance measurement hosts installed on research and public networks around the world. By using BBR to send data from one of ESnet’s test perfSONAR hosts, engineers can get details on the data transfer performance across 2,000 network paths with 2,000 different characteristics. “perfSONAR provides us with a really valuable test environment for new networking innovations like TCP BBR,” Tierney said.
Tierney said the old analogy of the Internet as an information highway really applies to the current version of TCP. As freeway traffic gets more congested, even the slightest event like a car with a flat tire can dramatically compound the situation as everyone slows down to rubberneck or go around.
With the current version of TCP, data traffic slows down when it hits congested stretches of a network and the problem gets worse the farther data travels. On some paths, BBR appears to help mitigate this slow-down.
ESnet is particularly interested in BBR as the network carries massive sets of data between 50 main sites in the U.S. and across four high-speed trans-Atlantic links.
“It looks like anyone who transfers big data over long distances could see huge improvements,” Tierney said. “The longer the path they are covering the bigger the difference will be.”
Watch the ESnet blog for updates as BBR tests are conducted. Results will also be shared on ESnet’s Fasterdata website where the organization houses best practices and a broad knowledgebase focused on high performance network engineering, performance testing and tuning, among other topics.