Teraflow Testbed: Angle Anomaly Detection Project
Website
http://angle.ncdm.uic.edu/http://sc07.supercomputing.org/html/SC07AwardWinners.html
Collaborators
Consortium for Data Analysis Research (CDAR), consisting of the UIC National Center for Data Mining (NCDM); University of Chicago Computation Institute; Northwestern University International Center for Advanced Internet Research (iCAIR); Argonne National Laboratory Mathematics and Computer Science Division; Toyota Technological Institute at Chicago
While currently a US-based effort, it does utilize the Teraflow Testbed, which is international in scope.
Description
Because the Internet is distributed, so is the data that must be analyzed to protect it. With today’s supercomputers, the data must be collected, transported to the supercomputer, and then transported back. For large data, the time required to do this can be a significant fraction of the total time required by the analysis. One of the innovations of the Angle project is the use of a data and compute cloud so that the data can be left in place and computation performed over the data.
Although cloud computing has been used for the past several years by companies such as Google, Yahoo, Amazon and Microsoft to provide their services, these cloud infrastructures, by and large, are based on the standard Internet. In contrast, the Sector data cloud, used by the Angle project, is a second-generation data cloud based on wide-area high-performance networks. These high-performance networks enable the large datasets produced by the project to be handled easily.
The Angle project was sponsored in part by the Consortium for Data Analysis Research, a Chicago-based research consortium that is developing new technologies and methodologies for analyzing large, complex and distributed data. NCDM won two of the first three Analytic Challenges (at SC05 and SC07).