NERSC, ESnet and JGI Explore Hardware as a Service

The National Energy Research Scientific Computing (NERSC) Division, Energy Sciences Network (ESnet) and Joint Genome Institute (JGI) facilities at the Lawrence Berkeley National Laboratory (Berkeley Lab), are showing that a "hardware as a service" model, where computing resources are provisioned as needed, could be quickly deployed to meet the computing demands of the research community.

In March 2010, JGI became an early user of NERSC's Magellan cloud computing system when the facility had a sudden need for increased computing resources. In less than three days, NERSC and JGI staff provisioned and configured hundreds of processor cores on the Magellan system to match the computing environment available on JGI's local compute clusters. At the same time, staff at both centers collaborated with ESnet network engineers to deploy a dedicated 9Gbps virtual circuit between NERSC's Magellan system and JGI over ESnet's Science Data Network within 24 hours.

According to Jeff Broughton, who heads the NERSC Division's Systems Group, this strategy gives JGI researchers around the world increased computational capacity without any change to their software or workflow. JGI users still log on to the Institute's network and submit scientific computing jobs to its batch queues that are managed by hardware located in Walnut Creek. Once the jobs reach the front of the queue, the information travels 20 miles on reserved SDN bandwidth, directly to NERSC's Magellan system in Oakland, Calif. After a job has finished, the results are sent back to Walnut Creek on the SDN within milliseconds to be saved on filesystems at JGI.

"This solution would not be possible without a reliable, high-bandwidth research network like ESnet's SDN. Although NERSC and JGI are 20 miles apart, Magellan is linked to JGI's infrastructure on a single, secure, dedicated network connection. The information travels as if these systems are in the same room and connected by a local area network," says Brent Draney, head of NERSC’s Networking, Security and Servers Group.

"What makes this use of cloud computing so attractive is that JGI users do not notice a difference between computing on Magellan, which is 20 miles away at NERSC, or on JGI's computing environment in Walnut Creek," says Broughton.

For more information about NERSC and JGI partnerships, please visit:

NERSC and JGI Join Forces to Tackle Genomics HPC