MIO: Tailoring Multi-level Decomposition for Storage I/O for Computational Applications

As computational sciences usher in a new era with vastly improved model fidelity empowered by the recent advances in high-performance computing (HPC), there is an urgent need to re-design the data analytics toolchain in the commercial market so that it can better adapt to the rapid growth of data coming out of large-scale simulations and maintain a high level of productivity. To address this need, this project proposes to develop a multi-level data retrieval library for hierarchical storage systems, so that users can request the minimum amount of data from fast storage tiers and pay higher input/output overheads only when needed. This project aims to address Focus Area 13 Information Technologies for Science Data to provide S17.01: Technologies for Large-Scale Numerical Simulation. Through co-designing the decomposition and recomposition over hierarchical storage, this project will offer new capabilities to the user communities, including those in the federal market (e.g., GEOS-5, FUN3D developers) and private sectors, and vastly improve the efficiency and flexibility of data analysis.
Benefits: The project team has engaged potential users in NASA (Sharad Gavali, Eric Nielsen for CFD Fun3D) and discussed various use cases of multilevel data representation. MIO will allow applications in NASA to prioritize those data that are more important to data analysis, thus resulting in significantly improved end-to-end performance. • For earth system modeling, such as GEOS-55 at NASA, there is a general desire to improve the resolution configuration, so that scientists can simulate the earth system with higher fidelity. In particular, scientists are interested in the spatiotemporal domains that contain significant climate events, such as the atmospheric river and tropical cyclones, and calculate new derived quantities, such as the total precipitable water vapor for a given geographical region. MIO will allow for multi-level accuracy for the simulation data—for those timesteps/regions that contain the extreme events, higher accuracy can be provided by MIO for more accurate prediction, while for other timesteps/regions, lower accuracy may suffice to reduce the I/O cost. • Computational fluid dynamics (CFD): Users of Fun3D may examine particular regions/timesteps in a simulation. For example, Fun3D was used to study aeroacoustics forces over the thermal protection system in a Space Launch System. Similarly, for aerodynamics, drag and lift forces on a wing during take-off/landing are often calculated. For both these cases, MIO will allow for multi-level accuracy of data to capture the important physics for regions of interest while reducing the data overhead. Similarly, the project team has been working with the following non-NASA users, Jackie Chen and Kisung Jung for combustion S3D, Michael Churchill for fusion XGC3, Qian Gong and Jill Zhang for climate modeling E3SM. The team has also worked with commercial applications, such as NUMECA (now part of Cadence, CFD simulation), Ramgen Power Systems (turbine design), FM Global (fire modeling), and ITER (fusion energy), in the past. The project team will re-engage these vendors to adopt MIO.