ISC'14

June 22–26, 2014
Leipzig, Germany

Presentation Details

 
Name: Fast Multi-Resolution Reads of Massive Simulation Datasets
 
Time: Wednesday, June 25, 2014
02:45 pm - 03:15 pm
 
Room:   Hall 3
CCL - Congress Center Leipzig
 
Speaker:   Sidharth Kumar, University of Utah
 
Abstract:   Today's massively parallel simulation code can produce output ranging up to many terabytes of data. Utilizing this data to support scientific inquiry requires analysis and visualization, yet the sheer size of the data makes it cumbersome or impossible to read without computational resources similar to the original simulation. We identify two broad classes of problems for reading data and present effective solutions for both. The first class of data reads depends on user requirements and available resources. Tasks such as visualization and user-guided analysis may be accomplished using only a subset of variables with restricted spatial extents at a reduced resolution. The other class of reads require full resolution multi-variate data to be loaded, for example to restart a simulation. We show that utilizing the hierarchical multi-resolution IDX data format enables scalable and efficient serial and parallel read access on a variety of hardware from supercomputers down to portable devices. We demonstrate interactive view-dependent visualization and analysis of massive scientific datasets using low-power commodity hardware, and we compare read performance with other parallel file formats for both full and partial resolution data.

Authors
Sidharth Kumar, University of Utah; Cameron Christensen, University of Utah; Peer-Timo Bremer, LLNL; Eric Brugger, LLNL; Valerio Pascucci, University of Utah; John Schmidt, University of Utah; Martin Berzins, University of Utah; Hemanth Kolla, Sandia National Laboratory; Jacqueline Chen, Sandia National Laboratory; Venkatram Vishwanath, Argonne National Laboratory; Philip Carns, Argonne National Laboratory; Ray Grout, NREL