openPMD A FAIR, self-describing standard for particle-mesh data for the Exascale era


openPMD A FAIR, self-describing standard for particle-mesh data for the Exascale era

Pöschel, F.; Hübl, A.

openPMD is a community meta-data standard for self-describing, FAIR and compatible exchange of scientific data on top of modern file formats.

The openPMD-api has successfully established an ADIOS2 implementation for efficient and scalable I/O and recently stabilized support for streaming features of ADIOS2. This talk evaluates two use cases for the SST engine in scientific simulation workflows with an I/O stack built by openPMD-api and ADIOS2. The first use case aims for an ad-hoc implementation of asynchronous IO, while the second use case discusses a loosely coupled simulation-analysis workflow with massive data exchange, both evaluated on Summit using the Infiniband network (RDMA).

Keywords: high-performance computing; big data; ADIOS; openPMD; streaming; RDMA

  • Invited lecture (Conferences) (Online presentation)
    Workflow Systems Group Seminar, 22.07.2021, Oak Ridge, USA

Permalink: https://www.hzdr.de/publications/Publ-33689
Publ.-Id: 33689