On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective

On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective

Huebl, A.; Widera, R.; Schmitt, F.; Matthes, A.; Podhorszki, N.; Choi, J. Y.; Klasky, S.; Bussmann, M.

We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today's and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.

Keywords: I/O HPC data reduction compression scalability PIConGPU manycore

  • Contribution to proceedings
    The 1st International Workshop on Data Reduction for Big Scientific Data (DRBSD-1) co-located with ISC High Performance, 18.-22.06.2017, Frankfurt, Deutschland
    High Performance Computing, Vol. 10524, 15-19
    DOI: 10.1007/978-3-319-67630-2_2
    Cited 7 times in Scopus
  • Lecture (Conference)
    The 1st International Workshop on Data Reduction for Big Scientific Data (DRBSD-1) co-located with ISC High Performance, 18.-22.06.2017, Frankfurt, Deutschland

Downloads

Permalink: https://www.hzdr.de/publications/Publ-25403