Science DMZ for ALS

Many beamline scientists at Berkeley Lab’s Advanced Light Source (ALS) are or will be experiencing slower network speeds because of instrument upgrades. These new instruments, or more specifically detectors, are yielding immense data output rates on the scale of hundreds of megabytes per second, which are causing bottlenecks in many of the beamline workflows.

For example, at one of the X-ray tomography beamlines the scientists had an increase in data output by 50-fold when they upgraded their detector. As a result, the beamline’s data workflow became broken from the detectors to the workstations, and data needed to be exported via USB drives or other hardware methods.

Figure 1: Beamline workflow before the Science DMZ was implemented.

To improve the infrastructure of the workflow and data export, ESnet deployed a Science DMZ model with both hardware and software solutions. ESnet's first step used perfSONAR to diagnose problem areas in the network. From there they were able to reroute the beamline’s data acquisition line to go straight to their data transfer node (DTN), which reduced some data flow redundancies (see Figure 2). ESnet also deployed a server upgrade. Originally the beamline was equipped with a 5TB Windows file server, which was upgraded to a Linux server DTN equipped with Globus Online (a GridFTP data transfer tool) and 40TB of disks storage. By making the above changes and deploying Science DMZ components to the beamline's workflow, ESnet was able to accelerate the beamline performance up to 200MB/s.

Globus collaborators were key in helping the ALS scientists succeed in getting data from the experiment site to off-site locations, like the NERSC supercomputing center. ESnet engineers and Globus Online staff were able to set up an automated workflow for data analysis at NERSC, thereby accelerating the beamline’s productivity.

 

Figure 2: Beamline workflow after the Science DMZ components were implemented to improve workflow performance.