Workload-Intelligent Data Storage for Next-Generation Advanced Computing Centers
Scientific Area: Advanced Computing
Funding: UT Austin 100,000 USD
PIs:
UT — Amit Ruhela (TACC, UT Austin)
| PT — João Paulo (INESC TEC and University of Minho)
Start and End Dates: 2025-09-16 - 2026-12-31
Summary: The WISE project aims to revolutionize data management for advanced computing by developing the first generation of storage systems capable of predicting the intrinsic and dynamic I/O patterns of data-intensive workloads and self-tuning their configurations based on such patterns.
By improving the operational efficiency of supercomputers, this project is set to significantly increase the performance and speed at which advanced computing centers support ground-breaking research. However, two main open research questions make this a high-risk, high-reward proposal: i) how can one accurately and timely predict the relevant set of I/O patterns of thousands of heterogeneous and dynamic workloads running at a given supercomputer? and ii) based on the predicted I/O patterns, how can these be used to automatically fine-tune or even develop tailored optimizations that will improve the overall efficiency of supercomputers?
The project will seek to answer these two questions by leveraging the expertise of INESC TEC’s team on storage and AI systems alongside researchers from TACC and MACC with extensive experience in managing HPC infrastructures. The project’s results will be disseminated through top scientific venues and integrated into an open-source proof-of-concept prototype, laying the groundwork for advanced computing centers’ first workload-aware data storage solution.
Main Outcomes: As success indicators, WISE will consider a final report detailing the project’s research and dissemination outputs, along with two proof-of-concept open-source prototypes, including the prediction engine and integrated SDS solution.
Moreover, the project aims to submit at least two research papers to peer-reviewed workshops (e.g., REX-IO, HotStorage, FTXS, or PDSW) and to major HPC conferences (e.g., SC, ATC, HPDC, CCGrid, Cluster, IPDPS).
Finally, the project will include both MSc and PhD students contributing to their training and specialization in the HPC and storage systems fields.