The U.S. government-funded TeraGrid project has received the green light to move forward with another stage, one that calls for the Pittsburgh Supercomputing Center at Carnegie Mellon University and the University of Pittsburgh to join the grid, which already links computer centers at four other research facilities and universities.
A National Science Foundation grant totalling US$35 million has been awarded to fund the ETF, an expansion of the grid to include the Pittsburgh center, which will be tightly integrated with and linked over a 40G-bps (bit-per-second) network to the four other facilities.
When complete, the TeraGrid will include 20 teraflops of computing power distributed over the five sites, storage capacity of nearly 1 petabyte, high-resolution visualization environments and toolkits for grid computing, according to the National Center for Supercomputing Applications (NCSA) at the University of Illinois.
The National Science Foundation announced the $35 million in funding last month for the ETF, said Richard Hirsh, acting director of the NSF’s Division of Advanced Computational Infrastructure and Research, said Tuesday.
Last year NSF awarded a $53 million grant to the four other facilities — the NCSA, the San Diego Supercomputer Center, Argonne National Laboratory and the Center for Advanced Computing Research at the California Institute of Technology in Pasadena — to establish the Distributed Terascale Facility. The year before NSF awarded $36 million to the Pittsburgh center for its supercomputing project.
The ETF award will provide money to integrate the two computing environments to create an extended terascale-level grid of data, computation and visualization resources that will make possible new scientific discoveries, NCSA said in a release last month. The five sites will be linked by the world’s fastest dedicated optical research network, built in partnership with Qwest Communications International Inc. and designed to accommodate additional connections, the release said.
The expansion of the TeraGrid will provide computing power to scientists that is orders of magnitude beyond anything seen before, Dan Reed, director of NCSA and chief architect for the TeraGrid project, said in the release. It will provide the best high-resolution visualization environments and more storage capacity than has been possible in the past, he said, adding that the impact on scientific discovery will be significant.
When NSF took a good look at the TeraGrid infrastructures thus far in place, it realized it needed to make the TeraGrid extensible so that other scientists could tap into it, and the Pittsburgh center needed to be connected over the 40G-bit network, Hirsh said.
“What we are doing is trying to link together really high-end machines with a really high-end network,” Hirsh said. “The concept of having this fat a pipe connecting these machines is as impressive as the machines themselves. We will be able to transfer data across the country at an unheard of rate.”
Tying in the Pittsburgh center also represents the first test of the heterogeneity of the TeraGrid, Hirsh said, as IBM Corp. Linux clusters powered by Intel Corp. processors are integrated with Hewlett-Packard Co. servers using Alpha processors.
The NCSA is home to the bulk of the computers that power the TeraGrid. When completely installed, its TeraGrid system there will consist of 10 teraflops of computing capacity in IBM Linux clusters powered by Itanium 2 processors. The clusters include 2 teraflops of computing power already on the floor when the TeraGrid project began, and another 8 teraflops paid for by the NSF TeraGrid awards. The system at NCSA also includes 240 terabytes of secondary storage, NCSA said.
The Pittsburgh Supercomputing Center will provide more than 6 teraflops, including 0.3-teraflops from its HP Marvel system, creating a shared memory computing capability of 512 gigabytes of memory, and adding a 150-terabyte disk cache.
By January, NSF, which expects an additional $20 million in funding from Congress over the next two years for the TeraGrid project, will solicit the scientific community for further expansion proposals. The idea of linking even more sites to the grid, such as data sites, supercomputing sites or sites that track instruments used to collect data from distributed sensors, has been proposed, Hirsh said.
“We like to let the community use its own wisdom and imagination to come up with ideas,” he said. “But it must support high-end science and have a broader impact through education. We are hoping for some really clever ideas. There are obvious targets, but there may be some not so obvious targets as well.”
