ESnet, KINBER Peering to Enhance Network Capabilities to PA Universities

KINBER, the Keystone Initiative for Network Based Education and Research in Pennsylvania, and ESnet, the Department of Energy’s Energy Sciences Network, announced today (Nov. 18)  the establishment of a new peering connection between KINBER’s PennREN network and the ESnet national network via Internet2’s AL2S service.

The new 10 gigabit-per-second (Gbps) based peering connection will allow for greater capacity, speed and enhanced research capabilities for several PennREN connected higher education institutions across Pennsylvania, including Swarthmore College, Drexel University, Pennsylvania State University, Cheyney University of Pennsylvania, Carnegie Mellon University, University of Pittsburgh, and Lehigh University.

Research applications that will benefit from this new enhanced connection include those in areas such as exascale computing; machine learning algorithms for matching theories, simulations and observations in cosmology; chemical imaging and other advanced Department of Energy-funded basic research in physical science.

Read the full story.

Attending SC16? Check out these 2 high-speed demos


ESnet is collaborating with Stanford University and SLAC to present two presentations and demonstrations of high speed data transfer as part of the SC16 conference. If you’re in Salt Lake City, drop by and watch the data fly.Please also forward this invitation to others that may be interested.

3-4 p.m. Tuesday, Nov. 15, in  the Stanford/SLAC booth #2101 we will be presenting three talks on:

  1. The science (Linear Coherent Light Source-II) needs to transfer regularly massive amounts of data (1 Tbit/sec by 2024) from the experiment in Menlo Park (SLAC) to an Exascale computer in Berkeley (NERSC)
  2. How ESnet is enabling distributed data intensive science and hence LCLS-II
  3. A demonstration of transferring Lots of Small Files data motivated by LCLS’s need for semi real-time transfer and access to the data acquisition system’s output.

11 a.m.-12 noon Wednesday, Nov. 16, in the Department of Energy booth #1030 we will be demonstrating high-speed file-to-file transfers.

This will include a cost effective, energy and space efficient, high availability forward looking software and data transfer system reference design for high speed file transfer. We will demonstrate:

  • Over the LAN file to file data transfer of 10 x100GiB files over 2 x 100Gbps links.
  • Over the wide area (122ms link) with a 70Gbps ESnet OSCARS shared circuit including Lots of Small Files  (1 million  x 1MByte files) transfer with and without TLS encryption

ESnet’s Expertise Underpins SC16 Conference Events

Each year, more than 10,000 professionals in high performance computing, networking, data storage and analysis convene from around the world at the SC conference. For six days, the latest innovations, technologies and ideas will be shared during SC16 in Salt Lake City, with significant support from ESnet.

ESnet’s expertise in networking will be shared in workshop sessions, talks, demonstrations and overall support for the conference. In particular, This year, ESnet is helping provision 600 Gbps of bandwidth (6 x 100G) between ESnet and the SC16 conference, which is more than the network has ever brought up to SCinet in the past.

Read more about ESnet-related activities at SC16.

Here’s a look at SC16 demos supported by ESnet:



Working Group on Open Science Cybersecurity Risks Releases First Document Draft for Public Comment

Michael Dopheide (left) and Sean Peisert (right).

Over the past several months, ESnet and the NSF Cybersecurity Center of Excellence collaborated with research and education community leaders to develop a risk profile for open science to formally capture and benchmark this expertise, allowing other organizations to apply these best practices more broadly.

Today, the group is releasing its draft Open Science Cyber Risk Profile (OSCRP) and inviting comment from the research community. The OSCRP is designed to help principal investigators and their supporting information technology professionals assess cybersecurity risks related to open science projects. The draft document, along with information on how to comment, can be found at

Managing the security risks to scientific instruments, data and cyberinfrastructure is a priority  for creating a trustworthy environment for science. Assessing, understanding and managing concerns of open science to explicitly capture risks to its integrity and availability, and sometimes also privacy issues, involves making judgments on the likelihood and consequences of risks. Deep experience in understanding cybersecurity and the science being supported is needed to achieve these goals.

The group invites comments on the document prior to final publication in early 2017.  Longer-term, the document is intended to be a living, community document, being updated as open science computing evolves, and also as new approaches to security arise.

About the OSCRP Working Group

Organized by Sean Peisert and Michael Dopheide from ESnet, and Von Welch and Andrew Adams from the NSF Cybersecurity Center of Excellence, the working group consists of: RuthAnne Bevier (Caltech), Rich LeDuc (Northwestern), Pascal Meunier (HUBzero), Stephen Schwab (USC Information Sciences Institute) and Karen Stocks (Scripps Institution of Oceanography), Ilkay Altintas (San Diego Supercomputer Center), James Cuff (Harvard), Reagan Moore (iRods), and Warren Raquel (NCSA/UIUC). To follow the activities of the working group, please follow

About the NSF Cybersecurity Center of Excellence •  

The Center for Trustworthy Scientific Cyberinfrastructure (CTSC) is funded as the National Science Foundation’s Cybersecurity Center of Excellence. The mission of CTSC is to improve the cybersecurity of NSF science and engineering projects, allowing those projects to focus on their science endeavors. This mission is accomplished through one-on-one engagements with projects to address their specific challenges; education, outreach, and training to raise the state of security practice across the scientific enterprise; and leadership on bringing the best and most relevant cybersecurity research to bear on the NSF cyberinfrastructure research community.

About ESnet •

The Energy Sciences Network (ESnet) is an international, high-performance, unclassified network built to support scientific research. Funded by the U.S. Department of Energy’s Office of Science (SC) and managed by Lawrence Berkeley National Laboratory, ESnet provides services to more than 40 DOE research sites, including the entire National Laboratory system, its supercomputing facilities, and its major scientific instruments. ESnet also connects to over 140 research and commercial networks, permitting DOE-funded scientists to collaborate productively with partners around the world.

National Science Foundation & Department of Energy’s ESnet Launch Innovative Program for Women Engineers

Women in Networking @SC (WINS) Kicks off this week in Salt Lake City!

WINS Participants
(Left to Right) Julia Locke (LANL), Debbie Fligor (SC15 WINS returning participant, University of Illinois at Urbana-Champaign), Jessica Schaffer (Georgia Tech), Indira Kassymkhanova (LBNL), Denise Grayson (Sandia), Kali McLennan (Univ. of Oklahoma), Angie Asmus (CSU). Not in photo:  Amber Rasche (N. Dakota State) and Julie Staats (CENIC).

Salt Lake City, UT – October 26, 2016 – The University of Corporation for Atmospheric Research (UCAR) and The Keystone Initiative for Network Based Education and Research (KINBER) together with the Department of Energy’s (DOE) Energy Science Network (ESnet) today announce the official launch of the Women in Networking at SC (WINS) program.

Funded through a grant from the National Science Foundation (NSF) and directly from ESnet, the program funds eight early to mid-career women in the research and education (R&E) network community to participate in the 2016 setup, build out and live operation of SCinet, the Supercomputing Conference’s (SC) ultra high performance network. SCinet supports large-scale computing demonstrations at SC,  the premier international conference on high performance computing, networking, data storage and data analysis and is attended by over 10,000 of the leading minds in these fields.

The SC16 WINS program kicked off this week as the selected participants from across the U.S., headed to Salt Lake City, the site of the 2016 conference to begin laying the groundwork for SCinet inside the Salt Palace Convention Center. The WINS participants join over 250 volunteers that make up the SCinet engineering team and will work side by side with the team and their mentors to put the network into full production service when the conference begins on November 12. The women will return to Salt Lake City a week before the conference to complete the installation of the network.

“We are estimating that SCinet will be outfitted with a massive 3.5 Terabits per second (Tbps) of bandwidth for the conference and will be built from the ground up with leading edge network equipment and services (even pre-commercial in some instances) and will be considered the fastest network in the world during its operation,” said Corby Schmitz, SC16 SCinet Chair.

The WINS participants will support a wide range of technical areas that comprise SCinet’s incredible operation, including wide area networking, network security, wireless networking, routing, network architecture and other specialties. 

Several WINS participants hard at work with their mentors configuring routers & switches

“While demand for jobs in IT continues to increase, the number of women joining the IT workforce has been on the decline for many years,” said Marla Meehl, Network Director from UCAR and co-PI of the NSF grant. “WINS aims to help close this gap and help to build and diversify the IT workforce giving women professionals a truly unique opportunity to gain hands-on expertise in a variety of networking roles while also developing mentoring relationships with recognized technical leaders.”

Funds are being provided by the NSF through a $135,000 grant and via direct funding from ESnet supported by Advanced Scientific Computing Research (ASCR) in DOE Office of Science. Funding covers all travel expenses related to participating in the setup and operation of SCinet and will also provide travel funds for the participants to share their experiences at events like The Quilt Member Meetings, Regional Networking Member meetings, and the DOE National Lab Information Technology Annual Meeting.

“Not only is WINS providing hands-on engineering training to the participants but also the opportunity to present their experiences with the broader networking community throughout the year. This experience helps to expand important leadership and presentations skills and grow their professional connections with peers and executives alike,” said Wendy Huntoon, president and CEO of KINBER and co-PI of the NSF grant.

The program also represents a unique cross-agency collaboration between the NSF and DOE.  Both agencies recognize that the pursuit of knowledge and science discovery that these funding organizations support depends on bringing the best ideas from people of various backgrounds to the table.  

“Bringing together diverse voices and perspectives to any team in any field has been proven to lead to more creative solutions to achieve a common goal,” says Lauren Rotman, Science Engagement Group Lead, ESnet. “It is vital to our future that we bring every expert voice, every new idea to bear if our community is to tackle some of our society’s grandest challenges from understanding climate change to revolutionizing cancer treatment.”

2016 WINS Participants are:

  • Denise Grayson, Sandia National Labs (Network Security Team), DOE-funded
  • Julia Locke, Los Alamos National Lab (Fiber and Edge Network Teams), DOE-funded
  • Angie Asmus, Colorado State (Edge Network Team), NSF-funded
  • Kali McLennan, University of Oklahoma (WAN Transport Team), NSF-funded
  • Amber Rasche, North Dakota State University (Communications Team), NSF-funded
  • Jessica Shaffer, Georgia Institute of Tech (Routing Team), NSF-funded
  • Julia Staats, CENIC (DevOps Team), NSF-funded
  • Indira Kassymkhanova, Lawrence Berkeley National Lab (DevOps and Routing Teams), DOE-funded

The WINS Supporting Organizations:
The University Corporation for Atmospheric Research (UCAR)

The Keystone Initiative for Network Based Education and Research (KINBER)

THe Department of Energy’s Energy Sciences Network (ESnet)

ESnet Talks Science Cyberinfrastructure with NSF and Tweet the Quilt

ThESnet_Final_Logos_All_Blue_Circle_Stamp_RGBis month, the Department of Energy’s ESnet co-located its bi-annual Site Coordinators Committee (ESCC) meeting with The Quilt’s Fall Meeting, as well as the National Science Foundation’s (NSF’s) Campus CyberInfrastructure and Cybersecurity Innovation for Cyberinfrastructure P.I. Workshops at the Marriott Downtown in Philadelphia, Penn. Holding all four meetings in one location allowed engineers and researchers to have fruitful discussions that highlight areas of science and related cyberinfrastructure that are important to the combined communities.

The co-located agenda included panel discussions on topics such as:

  • “National Cybersecurity,” facilitated by Berkeley Lab’s Sean Peisert. ESnet’s Michael Sinatra was a panelist.
  • “Science Engagement – Bridging the Gaps,” facilitated by ESnet’s Lauren Rotman. Berkeley Lab’s Craig Tull was a panelist.
  • “Climate Science Infrastructure Support,” facilitated by Colorado State University’s Christos Papadopoulos. Travis O’Brian from Berkeley Lab’s Earth Sciences Area was a panelist.

The meeting was held October 18-21, 2016.

Inder Monga Gives Plenary Talk at CHEP 2016

ESnet Director Inder Monga opened the Tuesday, Oct. 11 plenary session at the  22nd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2016, giving a talk on “Making Way for the Elephants.” View Monga’s slides.


Monga also participated in a in a panel discussion as part of a session on “Diversity and Bias — are we really unbiased?”

Monga and ESnet’s Eli Dart also served on the CHEP 2016 Local Organizing Committee.

Read more about Berkeley Lab’s contributions to the conference.

Berkeley Lab Collaboration with Ciena Enhances Optical Chip Design Process

A unique collaboration between a U.S. telecommunications equipment provider and a Department of Energy (DOE) Office of Science national laboratory has helped dramatically improve design cycle times for future high-speed optical networking components.

wl3-extreme-chipComputer scientists and mathematicians from Lawrence Berkeley National Laboratory’s (Berkeley Lab) Computing Sciences organization worked with engineers at Ciena, a leading networking company, to speed up the process by which Ciena validates the design of its ASIC (application-specific integrated circuit) chips. The collaboration grew out of the existing relationship between Ciena, a pioneer in high-bandwidth optical transport technology, and the DOE’s Energy Sciences Network (ESnet), which uses Ciena products to support its high-speed network.

“Modern science requires fast global networks like ESnet, and almost all DOE network traffic happens to run over optical gear from Ciena,” said ESnet Director Inder Monga. “ESnet doesn’t transport quite as much traffic as, say, Google, but we have the unique challenge of coping with massive single data streams from detectors and instruments like the Large Hadron Collider, the world’s largest and most powerful particle accelerator. This means we need to exploit leading-edge networking components, and we’re often early adopters of those components.”

Read more.

ESnet Begins Testing TCP-BBR

Update to underlying internet protocol promises to improve data transmission performance

With Google’s Sept. 16 release of the new TCP congestion control algorithm, BBR (Bottleneck  Bandwidth and Round Trip Time/RTT), ESnet staff are gearing up to test this promising development.

According to early reports, Google engineers reported that the new algorithm “significantly increased throughput and reduced latency.” Google cited an increase in the speed of data transfers from three megabits per second (Mbps) to 9,150 Mbps on a test link with 1% packet loss and a latency of 100 milliseconds. Early testing by ESnet engineers has shown improvements from 10x to 100x on some international paths, but no improvements on other paths.

“I’m eager to work with our Google collaborators to understand the details of how BBR works and how it may benefit data-intensive science applications in the future,” said Brian Tierney, head of ESnet’s Advanced Network Technologies Group and co-creator of ESnet’s Fasterdata website. “This is potentially a great step forward for TCP, and we’re eagerly testing BBR TCP. We’ll post results as we get them.”

More technical details will be revealed when a description of BBR appears in the September-October 2016 issue of ACM Queue. Among the developers of BBR is Van Jacobson, who was a primary contributor to the original TCP in the 1980s when he worked at Lawrence Berkeley National Lab, where he led the Network Research Group.

Tierney said ESnet has begun testing BBR using the 2,000 instances of the perfSONAR network performance measurement hosts installed on research and public networks around the world. By using BBR to send data from one of ESnet’s test perfSONAR hosts, engineers can get details on the data transfer performance across 2,000 network paths with 2,000 different characteristics. “perfSONAR provides us with a really valuable test environment for new networking innovations like TCP BBR,” Tierney said.

Tierney said the old analogy of the Internet as an information highway really applies to the current version of TCP. As freeway traffic gets more congested, even the slightest event like a car with a flat tire can dramatically compound the situation as everyone slows down to rubberneck or go around.

With the current version of TCP, data traffic slows down when it hits congested stretches of a network and the problem gets worse the farther data travels. On some paths, BBR appears to help mitigate this slow-down.

ESnet is particularly interested in BBR as the network carries massive sets of data between 50 main sites in the U.S. and across four high-speed trans-Atlantic links.

“It looks like anyone who transfers big data over long distances could see huge improvements,” Tierney said. “The longer the path they are covering the bigger the difference will be.”

The announcement about BBR was posted on the Patchwork Linux network development website and can be found at

Watch the ESnet blog for updates as BBR tests are conducted. Results will also be shared on ESnet’s Fasterdata website where the organization houses best practices and a broad knowledgebase focused on high performance network engineering, performance testing and tuning, among other topics.

2016 TechEx Conference Taps ESnet’s Expertise

From determining the future networking needs of researchers to speeding up the flow of scientific data, ESnet staff will share their expertise at the 2016 Technology Exchange being held Sept. 25-28 in Miami. The 2016 edition of TechEx is presented by Internet2 and co-hosted by Florida International University, The University of Florida and Florida LambdaRail.

On Sunday, Sept. 25, Jason Zurawski and Kate Mace of ESnet’s Science Engagement Team led “Demystifying the Science Requirements Review Process for Networking,” a half-day tutorial on the ESnet network requirements review process in the context of the Research and Education (R&E) networking community.

Brian Tierney, head of the Advanced Network Technologies Group, is one of three speakers presenting an overview of the newly released perfSONAR 4.0 network measurement software tool on Tuesday, Sept. 27.

Andy Lake of the Advanced Network Technologies Group will present a “perfSONAR 4.0 Feature Tour,” highlighting many of the important new additions and enhancements found in perfSONAR 4.0. It will cover topics such as improvements to MaDDash, updated test result charts, new central test configuration features, and information about OS upgrades.

Tierney will give another talk on Tuesday on “Improving Performance of 40G/100G Data Transfer Nodes,” which will cover host tuning for 40G/100G hosts, as well as new TCP enhancements that can greatly improve the stability of high-speed flows.