Meet Todd Anderson, ESnet’s New Director of Systems & Software

Todd Anderson may be new to his role as thTodd Croppede Energy Sciences Network’s (ESnet) Director of Systems and Software Engineering, but he’s already made history.

Anderson is the first ESnet staff member to be hired and onboarded completely virtually. And because of the Bay Area’s extended shelter-in-place order, he will be spending his first months on the job managing his team remotely from home in Lafayette, California.

Before coming to ESnet, he spent 20 years working at the executive level of a technology company that provided software-as-a-service (SaaS) solutions to financial institutions, including services in account identity risk management, fraud prevention, and digital payments, specifically the Zelle payment platform.

“It was a good gig. We were preventing fraud, but at the end of the day, our mission was to help big banks optimize their bottom line,” said Anderson. “I have an engineering background and I wanted to do something a little more meaningful. So, I looked for organizations around the Bay Area working in the fields of sustainability, cleantech, and renewable energy. That’s when I saw this opportunity at ESnet.”

At ESnet, he gets to apply his experience to manage the teams that develop and deploy tools to allow scientific users to optimize their use of the network.

“It’s really exciting to hear about the science projects I will be supporting as a member of the ESnet staff,” said Anderson. “And ESnet is doing its own networking research, too. It’s cool to be on a conference call and hear people talking about quantum networking and 5G.”

As a child during the space race, Anderson remembers watching the Apollo and space shuttle missions. That experience inspired him to major in mechanical engineering at the University of Colorado, Boulder, so he could one day build spacecraft for NASA. But after graduation, life took a different turn. Following in the footsteps of a friend he admired, he joined the Peace Corps and spent two years teaching math and science to middle school children in Botswana’s Kalahari Desert.

“Service really makes you feel like you are doing something worthwhile and making the world a better place,” said Anderson.

When he returned to the United States, a friend asked him to help write a software application to detect merchant credit card fraud. This move kicked-off of his 30-year career in technology.

In his free time, Anderson enjoys doing things around the house. When the world isn’t in the midst of a pandemic, he likes to be out in nature and to sample the diverse culture and geography of the Bay Area with his family.

Written by Linda Vu

ESnet Builds Morale and Community With a Zoom Competition

Nearly two months into California’s shelter-in-place order, we’ve all been in more than our fair share of video conferences. To boost morale during this difficult time, the Energy Sciences Network (ESnet) staff held a Zoom Background Competition during their all-to-all meeting on Monday, April 27. 

Staff were encouraged to create their own backgrounds and display them during the meeting. There were 21 entries. ESnet employees voted. Submissions were judged on overall artistry, functionality (not too distracting as a background), whether it elevated the voter’s mood, and if it made them feel included in the ESnet community. 

The top three winners got bragging rights. Here they are:

First place: Jeff Berman, NOC Engineer

This Zoom challenge inspired Berman, an avid sailoJeffrey Bermanr, to take to the sea. He won this competition with an hour video of the San Francisco skyline, one he filmed while sailing on the Bay. Although he typically likes to go sailing with friends and family, he says that sailing solo brings him a sense of peace, calm, and tranquility.  

“What is sailing? Most books define it as hours of sheer boredom scattered with white knuckle periods of terror. On a good day, both are true. Both give you an equal sense of accomplishment. How to be with yourself with nothing to do, good training for our current situation,” said Berman.

Second Place: Sartaj Baveja, Software Engineer

This challenge inspired Baveja to create a background meme of office life. In the background, someone (Baveja) is looking over your shoulder to catch a glimpse of your screen and make sure you don’t procrastinate.

Sartaj Baveja

Third Place: Joe Metzger, Network Engineer

Joe_interviewThis challenge inspired Metzger to use a picture that he took in Barcelona. The focal point of the picture (the blur) is a little girl in a red coat, black dress and white tights who was just running back and forth between the pools of light and shadow created by the stone arches and rosette windows, while her family was sitting in the cafe. 

“I used this as my zoom background because I think it is a really cool picture. It brings to mind a fun evening strolling around the little squares and back streets in Barcelona and sitting in cafes with a good glass of wine relaxing,” said Metzger.

Girl in Red

Written by Linda Vu, Berkeley Lab Computing Sciences.

ESnet’s iperf3 Tool Helps Comcast Meet Customer Demand During COVID-19 ISP Surge

A network diagnostic and performance measurement tool developed by engineers at the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) and Lawrence Berkeley National Laboratory (LBNL) is being used by Comcast to fine-tune the largest residential Internet network in the United States and help ensure its services remain up and running at top speed during the COVID-19 crisis.

“We are seeing an unprecedented shift in network usage, but it’s within the capability of our network,” Comcast states on its COVID-19 Network Update webpage.

Prompted by shelter-in-place orders across the U.S. and the world in recent months, extraordinary demand is being placed on residential Internet service providers (ISPs) as people increasingly rely on home-based Internet connections for entertainment, education, shopping, and work.

ESnet’s open-source iperf3 tool is helping Comcast meet this challenge by giving them the ability to make timing and buffering changes across their network in real-time. Originally developed in 2009 as part of the perfSONAR toolkit, iperf3 is designed to measure the available network bandwidth between two hosts on an IP network. It supports tuning of features related to timing, protocols, and buffers; for each test, it reports the measured throughput, loss, and other parameters.

At present, Comcast – which is seeing a 30% uptick in network traffic since the country’s shelter-in-place orders took effect – is using iperf3 to run 700,000+ diagnostic speed tests per day. This helps them engineer the network for peak capacity and better handle spikes and shifts in usage patterns.

“Comcast is one of the largest ISPs in the world, and they are using iperf3 – part of their normal troubleshooting workflow – to make sure their network is delivering the performance that is required in this situation,” said Bruce Mah, a software engineer in ESnet’s Software Engineering Group who works with Comcast as part of their open-source relationship.

ESnet, a DOE Office of Science user facility managed by LBNL, is the fastest network in the world dedicated to science. It supports a multi-100Gbps fiber optic backbone that connects the DOE’s national laboratory system and experimental facilities with research and commercial networks around the globe.

ESnet, NERSC Continue to Deliver Supercomputing, Networking Support for Nation’s Scientists during Pandemic


While people around the world hunker down in their homes to try to slow the advance of the COVID-19 virus and many services have decreased or stopped, two user facilities operated by the U.S. Department of Energy’s Office of Science continue to provide critical computing and networking resources to thousands of scientists, including some who are exploring ways to fight the pandemic.

The National Energy Research Scientific Computing Center (NERSC) and the Energy Sciences Network (ESnet) are managed by Lawrence Berkeley National Laboratory, which has reduced operations and onsite staffing under state-wide shelter-in-place orders. But NERSC and ESnet, deemed to provide essential services to the nation, continue to support “science as usual” as staff remotely manage the facilities from their homes.

NERSC has been named to the COVID-19 High Performance Computing Consortium. Led by the White House Office of Science and Technology Policy, industrial partners, and DOE, the consortium will give researchers access to supercomputers at DOE’s Argonne, Lawrence Livermore, Los Alamos, Oak Ridge, and Sandia national laboratories. ESnet will provide robust, high-bandwidth connections and peerings allowing scientists to tap into these computing resources and move data from across the world to those sites for analysis.

With an eye on pandemic-related research, NERSC staff have set up dedicated priority queues to run COVID-19-related research projects on a supercomputer. In one project, scientists at the Beckman Research Institute at the City of Hope are running molecular dynamic simulations that apply to a range of COVID-19 research areas. In particular, they are looking at the difference between the Chinese and Italian strains of the virus as well as potential antiviral treatments.

“It’s very challenging for everyone, it’s unprecedented,” said NERSC Division Director Sudip Dosanjh. “Our staff are very dedicated, and I think this also shows their passion for the science mission of NERSC, ESnet, and the laboratory.”

In fact, shelter-in-place policies across the country appear to be fostering even greater demands on the supercomputers at NERSC. With travel plans and conferences delayed or canceled, many of NERSC’s more than 7,000 users are spending their time at home but still want to advance their research by running projects on the center’s systems, Dosanjh surmised.

“Thanks to the dedicated efforts of NERSC personnel to keep computing systems running and supporting users’ requests, our ‘computing lab’ (NERSC) remains open and operational at full capacity,” said Manos Mavrikakis, a NERSC user and distinguished professor at the University of Wisconsin-Madison whose work focuses on understanding catalytic process principles and the discovery of new materials that would enable more efficient energy production. “As a result, we have been able to continue pursuing our research on catalytic reaction mechanisms, pretty much at the same pace as before coronavirus dominated everybody’s lifestyle. We are enormously grateful to NERSC personnel for an excellent job under highly stressful conditions.”

“We recognize the importance of that and are seeing that the utilization of Cori, our primary computer, is at 97 percent, an all-time high,” Dosanjh said. “A lot of other people can’t do their work unless we do our job, and I couldn’t be more proud of our staff.”

Cori, a Cray XC40 supercomputer able to perform nearly 30 quadrillion calculations per second, is used to create detailed models of scientific problems and analyze massive amounts of data from experimental facilities operated by DOE.

ESnet provides the critical high-bandwidth connection between tens of thousands of researchers at national labs, universities, user facilities and supercomputer centers like NERSC. ESnet operates a dedicated multi-100-gigabits-per-second network that crisscrosses the country and has four similar links crossing the Atlantic Ocean for collaborations in Europe. Almost all network traffic passing to and from DOE laboratories traverses the network.

Although ESnet’s operations center is in Berkeley, about 40 percent of the staff live in other states across four time zones and are used to working offsite. The network operates 24 hours a day, 365 days a year, enabling scientists to seamlessly access data portals, transfer massive research data sets, and tap into remote scientific instruments — all in real-time from anywhere.

The dispersed staff are both closer to other facilities and bring different perspectives to solving network issues, said Tony Ferrelli, head of ESnet’s Network Engineering and Operations Team. With so many people across the country working from home, ESnet has seen a blip in traffic moving onto the network, Ferrelli said, but there is still bandwidth to spare. One interesting note is that with more people working from home, they are finding that their home network connections are much slower than expected, which is compounded by increased demand, Ferrelli said.

Network staff is also on hand to help researchers should they need help to manage the large datasets that are typical of DOE science, ESnet Director Inder Monga said.

“It’s all about the people – those of us whose job is to provide these resources and those who tap into them to support our nation’s scientific leadership,” Monga said. “With all these efforts, science is proceeding as usual.”

By connecting with other research and education networks, ESnet is providing a critical link for scientists and consortium members like those from COVID-19 High Performance Computing Consortium with DOE supercomputer centers, thereby supporting research efforts into the COVID-19 pandemic.

Written by Jon Bashor

FABRIC project forms Scientific Advisory Committee


FABRIC, a project funded by the National Science Foundation, announces the formation of a Scientific Advisory Committee (SAC) tasked with facilitating collaboration and providing scientific and technological review for the project. FABRIC will create a unique national research infrastructure for testing novel architectures aimed at building an extensible, more secure Internet.

With leadership from the Renaissance Computing Institute (RENCI) at the University of North Carolina at Chapel Hill, the FABRIC project will build a large-scale platform with storage, computational and network hardware nodes across the country that are connected by dedicated high-speed optical links. FABRIC will also link major national research facilities such as universities, national labs and supercomputing centers that generate and process enormous scientific data sets.

The SAC will help guide the project by providing recommendations and critical feedback. Initially, the focus will be on reviewing the FABRIC design to ensure it can meet the diverse research needs of the future. The committee will also facilitate critical partnerships between collaborating institutions both within and outside of the US. As work progresses, the SAC will develop grand challenges that focus on solving key research problems using the FABRIC infrastructure.

“We are excited to have key research leaders across diverse career stages in fields such as networking, computing, software and security as our Scientific Advisory Committee,” said Inder Monga, co-PI of the FABRIC project and executive director of the U.S. Department of Energy’s Energy Sciences Network at Lawrence Berkeley National Laboratory. “The work is progressing well with FABRIC, and we look forward to the committee’s guidance on building an infrastructure that can facilitate testing of radical new ideas and approaches that will help lay the groundwork for the future Internet.”

FABRIC Scientific Advisory Committee Members are:

Sujata Banerjee, VMWare Research

Terry Benzel, University of Southern California

Kaushik De, University of Texas at Arlington

Cees de Laat, University of Amsterdam

Phillipa Gill, University of Massachusetts, Amherst

Abraham Matta, Boston University

Craig Partridge, Colorado State University

Jennifer Rexford, Princeton University

Scott Shenker, University of California, Berkeley

Frank Wuerthwein, University of California, San Diego

ESnet cybersecurity staff awarded the first-ever “ESNET”

Three members of the Energy Sciences Network’s Cybersecurity team – Michael Dopheide, Vlad Grigorescu, and Sam Oehlert – were recently honored with an Extra Special Noteworthy Exemplary Trophy Award for their SANS 2019 Holiday Hack Challenge entry.

“SANS is the premier security training organization for our profession. Their annual contest usually has 15,000-20,000 entries. For the past couple of years, our solutions have been used as the official answers and documentation, helping students and professionals around the world to hone their security skills. There is a real impact, and I am very proud of the team,” said Adam Slagell, Energy Sciences Network Chief Security Officer.

Dop Vlad Sam
From left to right: Michael Dopheide, Sam Oehlert, and Vlad Grigorescu

This eponymous award category called the “ESNET,” was created in reverence to the fact that all three team members asked to forgo any competition prizes. Instead, they asked that their prize be awarded to another winner.

“We felt that forgoing the prize was our small way of giving back to the community and rewarding one of the other participants, whose work should not be neglected. Honestly, this is something we look forward to every year. All three of us relish the puzzle-solving element, and our participation is really a labor of love,” said Grigorescu.  “We still felt it was important to share our report and the new techniques and tools for both offensive and defensive security we developed.”

The team notes that the SANS Holiday Hack competition is unique because the goal is to gamify cybersecurity challenges, all of which must be based on current threats. The end result is one of the best, most realistic security training, and it’s available for free, in perpetuity.

“By really pushing the envelope, we felt like we walked away from this year’s competition having significantly advanced our skills and being better able to safeguard the Energy Sciences Network. We’re incredibly honored to have an eponymous award, and to win the inaugural ‘ESNET’ award,” said Grigorescu.

This was Grigorescu’s fifth SANS Holiday Hack Competition and the fourth for Dopheide and Oehlert. All three were part of the Energy Sciences Network team that won the 2018 Grand Prize. Grigorescu’s teams also won an Honorable Mention in 2015, Best Technical Answer in 2016, and a Grand Prize in 2017.

Written by Linda Vu, Berkeley Lab Computing Sciences


Patty Giuntoli: Learning at the Speed of Life

Patty Giuntoli
Patty Giuntoli

On January 20, 2009, Patty Giuntoli was sitting in a human resources office at Berkeley Lab watching President Barack Obama being sworn in as the 44th President of the United States. Like him, she was starting a new job in public service too. 

Giuntoli was going to be the new director of infrastructure in Berkeley Lab’s IT division, managing the group in charge of Berkeley Lab’s local area network, desktop support, and telephone system. In 2011, she joined the Lab’s Scientific Networking division / Energy Sciences Network (ESnet) as the department head of networking and systems, where she helped design ESnet’s next-generation network, called ESnet6, and worked to shepherd it through the Department of Energy’s approval process into fruition.

Now after 11 years of Lab service, Giuntoli will be retiring on February 25.

“It wasn’t the mission that initially brought me to Berkeley Lab. I just saw the posting on LinkedIn and applied. But it was the mission that kept me here, longer than any other job in my career,” said Giuntoli. “One of the unique things about research and education that I really enjoy is the cross-collaboration. The ultimate focus is on a shared destiny – advancing science – and the mission transcends individual organizations.”

A second-generation San Franciscan, Giuntoli’s path to networking wasn’t straightforward. She received a B.S. in biophysics at the University of San Francisco and thought that she’d pursue a career in research. But the project that she hoped to work on after college didn’t receive the funding it needed, so she moved back home and looked for any job to bide time until another research opportunity.

Giuntoli would end up at GTE Lenkurt in San Carlos, working with microwave communications systems. This job marked the beginning of her 30 plus career in networking, which included stints as a telecom engineer at Bechtel, director at PG&E, VP at Oracle Corporation, and director of network services at Kaiser Permanente. Her management experience includes establishing and leading multi-functional, global organizations of over 500 people, developing and implementing business plans, technical and service strategies, and business and operational processes.

In addition to working on the ESnet6 project, Giuntoli notes that one of her greatest accomplishments at Berkeley Lab has been hiring new people into ESnet.

“I really enjoyed bringing new people in ESnet, hiring people from the outside that brought a different perspective and allowed us to get a mix of ideas. I liked putting mechanisms in place that encouraged people to think about things differently,” said Giuntoli. “This is actually what I’m going to miss most about working at ESnet – the people.”

Her advice to future generations? Act fast and don’t be afraid to fail.

“I see it as failing forward, a gift, you are learning something new every time. One thing that has been interesting to watch in research and education is how slow things move because people are waiting for things to be perfect. I say move faster and learn at the speed of life. Maybe one in 10 ideas will be a success, but that’s okay. Take a chance and put it out there,” said Giuntoli.

In retirement, Giuntoli hopes to spend more time with her grandchildren, playing golf, and painting landscapes.

Written by Linda Vu, Berkeley Lab Computing Sciences

ESnet Staff Participate in 2019 CC* PI, NRP and Quilt Meetings

ESnet’s Jason Zurawski, Lauren Rotman, and Eli Dart participated in three networking research meetings last month: the National Science Foundation’s Campus Cyberinfrastructure and Cybersecurity Innovation for Cyberinfrastructure PI (CC* PI) Workshop, The Quilt, and the Third National Research Platform Workshop. The event were held September 23 – 26 in Minneapolis, Minnesota.

Zurawski served on the planning committee for the CC* PI workshop and Dart gave a talk there entitled “Data Mobility Exhibition Motivations and Next Steps,” while Rotman served on the workshop program committee for the Third National Research Platform Workshop.

In addition, ESnet is co-PI with Indiana University on the Engagement and Performance Operations Center (EPOC), which is funded by NSF. During the CC* PI workshop, staff from ESnet and EPOC were involved in several events, including:

  • Led a half-day workshop with the University of Chicago/Globus entitled “The Data Mobility Workshop & Exhibition”; the focus of this event was to draw attention to the importance of a strategy for scientific data movement.
  • Led a session on “speed learning” entitled “Starting a Conversation with the Scientific Community and Strategies to Increasing Adoption & Awareness.”
  • EPOC led a panel discussion on “How Regional Partnerships with National Performance Engineering and Outreach Initiatives are Enabling Science.”

Meanwhile, during The Quilt meeting, EPOC gave a lightning talk on its scientific deep dives, which are similar to the ESnet requirements review and discuss scientific drivers and use of technology.

ESnet to Support Leading-Edge Demos at SC18

During SC18, ESnet staff will be working with other research organizations and vendors to support a series of demos at the forefront of networking, including the use of new hardware and software. These demos include:

  • SENSE: In collaboration with two universities and three national laboratories, ESnet will demonstrate its Software-defined Network for End-to-end Networked Science at Exascale research project.
Topology of the SENSE demo at SC18.
  • FAUCET: ESnet is collaborating with the University of Waikato in New Zealand and the faucet Foundation on a very large, never-before-done deployment of the Faucet controller, which was created to bring the benefits of software-defined networking to a typical enterprise network and has been deployed in various settings.
  • Monon400: As part of the Network Research Exhibition (NRE) at SC18, ESnet is collaborating with Indiana University (IU), Ciena and Internet2 to power Monon400, the fastest network ever built for research and education.

Read the full story.

Faucet – Enterprise SDN from an office to multi-terabit SCinet at SC18

This post is written by ESnet Director, Inder Monga, and shared through a personal lens, other perspectives may vary

I wanted to share a remarkable effort happening in the field of networking that is going to be shown in production at the Supercomputing 2018 as part of SCinet. Most of the people working in the SC booths (or even at SCinet) may never realize the role they are playing in moving the field of networking to the next level. Let me share some history [and you can skip the Background section to go straight to the main essence of the blog], and talk about the international, collaborative effort at play here.

Background: my software-driven journey

This is being shared by the perspective of the writer’s journey, other perspectives may vary

From the early 2000’s, the dream that excited me was building software layer that would manage networks and expose the network black-box as a set of ‘knobs and dials’ that applications could use to request and customize for their purposes. A small group of us at Nortel Networks came up with an idea of CO2 aka Content over Optics in 2001 – see the slide below.

This excitement was very new and led to my most productive set of patents filed during that time. Just as the concept was taking hold, an innovative research and education network, SURFnet, decided to deploy this concept on their newly built network, and the name evolved from CO2 to DRAC aka Dynamic Resource Allocation Controller.

Independently, ESnet was on a similar journey and started working on a concept called OSCARS aka ‘On-demand Secure Circuits and Reservation System’. As the community coalesced on the same direction, multiple initiatives like OSCARS, UCLP, DRAC, MANTICORE and others started merging towards an aligned software-driven network journey for the WAN. In the meantime, the Openflow effort from Stanford driven by an enterprise-based architecture, Ethane, captured the hearts of many, and the ‘Software-Defined Network’ (SDN) wave was born.

I am going to skip discussing the twists and turns of the evolution of SDN in industry, where there have been many recent articles on whether OpenFlow is dead. In fact there is a grassroots and under-the-radar SDN/Openflow effort that started in New Zealand called Faucet that proves these concepts are still alive and well, and are poised to thrive! In fact this work recently won the award in the New Zealand Open Source Software Project Award.


Faucet at ESnet

Ever since I was invited to join the Faucet Foundation Board, I was insistent that I ‘eat my own dog food’. With the help of Josh Bailey and Simeon Miteff, I began running a faucet controlled switch (OpenFlow is not dead) in my office as my only bridge into the Lab’s network and the Internet, i.e. if faucet did not work, I could not get connected! I am proud to say that other than one small self-inflicted upgrading glitch, the system has been working seamlessly over the past year no matter what I do or use in the office. You can see the couple of raspberry PIs running the SDN controller working with a commercial off-the-shelf switch in the image below.

[I wish it was cleaner picture, but I had to get everything together to click a photograph without my desk in the way.]

Then my colleague, Nick Buraglio, and I decided, why not wire our satellite branch office with faucet based networking than buy traditional switches? With Josh (Bailey)’s help, Nick has now wired the lives of eight of ESnet’s key employees to be dependent on faucet and he has been having a blast (no downtime there either)! Read more about his journey on his own private blog:

Faucet at SCinet, SC18

I may have led you on and will keep the suspense on for a bit longer….Faucet is being showcased in SCinet as production SDN controller managing a portion of the booth networks. SC18 is in two weeks and I encourage all of you to stop by SCinet to see it working! Reach out to the faucet champions – Josh Bailey, Brad Cowie, Richard Nelson, Nick Buraglio and Kate Mace who can talk you through the intricacies of the faucet deployment there. Poseidon, as you see printed on one of the raspberry pi’s in the image above, is also being featured at SC as it integrates learning and security with faucet.

More details on this topic will be available after SC is over. In the meantime, do check out Nick’s blog for the engineering details. An image from the recent multi-vendor plugfest at SCinet staging in Dallas to – yes, vendors support this.

Look forward to seeing you all in Dallas as all of us continue on this journey