ESnet, Globus Online and IU Demonstrate more Reliable Data Transfers

 A collaboration between the Department of Energy’s ESnet (Energy Sciences Network) and Globus Online, together with Indiana University (IU), is already producing proof-of-concept demonstrations of moving large scientific datasets.

As a lead-up to TIP2013, an international networking conference held from Jan. 13-18 in Hawaii, experts from ESnet, Globus Online and IU demonstrated the integration of three critical data transfer technologies:

• ESnet’s OSCARS (On-Demand Secure Circuits and Advance Reservation System) which provides multi-domain, high-bandwidth virtual circuits that guarantee end-to-end network data transfer performance.
• Globus Online’s application that provides easy-to-use, robust and secure file transfer capabilities – traditionally available only on expensive, special-purpose software systems – accessible to any researcher with an Internet connection and a laptop.
• Indiana University’s XSP (eXtensible Session Protocol) which enables scientific applications to automatically provision circuits or interact with software defined networks to meet the specific needs of the data transfer.

In a demo conducted in early January, the team was able to move one terabyte of data using a dynamically configured 5 gigabit per second (Gbps) OSCARS circuit between Argonne National Lab and Lawrence Berkeley National Lab in about 22 minutes. As a proof of concept, the demo achieved impressive transfer rates of just over 6 gigabits-per-second – highlighting how the integration of these technologies can greatly improve scientific productivity. The demo was conducted using ESnet-operated test GridFTP servers which are available for community testing.

In November, ESnet and Globus Online announced a collaboration to help scientists better manage the growing amounts of data they need to move, share and analyze worldwide. ESnet is the Department of Energy’s high performance science network that interconnects the nation’s research laboratories, supercomputing centers and experimental facilities. As an easy-to-use, highly reliable online file transfer and sharing service, Globus Online has become a primary on-ramp for researchers to access high performance networks like ESnet for rapidly sharing data or to use remote computing and scientific instrument facilities.  ESnet is also pursuing complementary research efforts with Indiana University.

Researchers at the Indiana Center for Network Translational Research and Education (InCNTRE) at IU have been developing XSP to transparently improve network performance in transferring bulk data. The XSP client library implementation provides a general interface for configuring dynamic network resources, including technologies such as OpenFlow and virtual circuits like OSCARS. Indiana University and Globus Online recently received a National Science Foundation Campus Cyberinfrastructure – Network Infrastructure and Engineering (CC-NIE) grant to accelerate the integration of XSP and GridFTP, and to bring the power and ease of use of Globus Online to the next-generation campus cyberinfrastructures being funded in other awards.

Raj Kettimuthu, a fellow at the Computation Institute, a joint initiative between The University of Chicago and Argonne National Laboratory, said the demonstration was aimed at bringing together three proven technologies to give researchers more predictability and reliability when moving the large datasets that are increasingly the hallmark of modern science. The Computation Institute is the driving force behind Globus Online.

Ian Foster, director of the Computation Institute, highlighted the results of the demonstration during his keynote presentation at TIP 2013. The meeting, which combines the twice-yearly Joint Techs meeting sponsored by ESnet and Internet2 with the Asia Pacific Advanced Network’s (APAN) meeting, drew more than 450 networking experts from around the world.

The challenge addressed by the demo is a common one faced by research and education network operators around the globe. As an example, Kettimuthu said, say five researchers are trying to move their data across a 10 gigabit-per-second link between Argonne and Lawrence Berkeley national labs. Under the current paradigm, the scientists using Globus Online would have access to “best effort” networking and would share the link with other users resulting in no guaranteed performance for their transfers.

OSCARS allows users to reserve a specified amount of bandwidth for a specified time, ensuring the data will be delivered with the highest performance possible. This allows the user to also schedule computational resources accordingly to perform the necessary simulations or analysis. Given the size of datasets and the overwhelming demand for computing time, such a capability is necessary. XSP provides the critical bridge to integrating Globus Online with resources like OSCARS to make this possible.

“Through this demonstration, we are showing the viability of high performance data transfer as a service – as a potential new model,” said IU Professor Martin Swany, director of InCNTRE and lead for the XSP research effort. “By automating every facet, including network performance tuning, dynamic network allocation and management of the ensemble end to end, we can improve data throughput and network efficiency. This demo is an important first step in our new NSF effort with the Computation Institute and a logical next step in our ongoing collaboration with ESnet.”