Within the high-performance computing field, the term “big data” has been used to describe datasets too large to be handled with on-hand analysis, processing, and visualization tools. It is anticipated that the ability to perform these fundamental tasks will become a key basis for competition and science discoveries within the near future. Recent advances in astronomical observing and simulation facilities are expected to move astronomy toward a new data-intensive era where such “big data” is the norm rather than an exception.To enable knowledge discovery in this new era, we designed and built the Tera-scale interactive visualization and data analysis framework (GraphTIVA). GraphTIVA is a high-performance, graphics processing unit (GPU)-based framework for the efficient analysis and visualization of Tera-scale 3-dimensional data cubes.
Using a cluster of 96 GPUs, we demonstrate for a 0.5 TB image: (1) volume rendering using an arbitrary transfer function at 7–10 frames per second; (2) computation of basic global image statistics such as the mean intensity and standard deviation in 1.7 s; (3) evaluation of the image histogram in 4 s; and (4) evaluation of the global image median intensity in just 45 s. These results correspond to a raw computational throughput approaching one teravoxel per second, and are 10–100 times faster than the best possible performance with traditional single-node, multi-core CPU implementations.
A scalability analysis shows the framework will scale well to images sized 1 TB and beyond. Other parallel data analysis algorithms can be added to the framework with relative ease, and accordingly, we present our framework as a possible solution to the image analysis and visualization requirements of next-generation telescopes, including the forthcoming Square Kilometre Array pathfinder radiotelescopes.