Software for Caching Output and Reads for Parallel I/O (SCORPIO)

The SCORPIO (Software for Caching Output and Reads for Parallel I/O) library was developed to improve the ability of component models of the Energy Exascale Earth System Model (E3SM) to perform I/O efficiently by leveraging other low-level I/O libraries. The SCORPIO library was derived from the Parallel I/O library (PIO) that has been developed over several years to improve the ability of component models of the Community Earth System Model (CESM) and Energy Exascale Earth System Model (E3SM) to perform I/O. However the generic interface provided by the library can be used by a wide range of structured and unstructured grid applications. SCORPIO includes support for many low-level I/O libraries like NetCDF, PnetCDF, HDF5 and ADIOS2.

SCORPIO continues to support the API supported by the PIO library and includes C and Fortran APIs compatible with the PIO library. The library also includes improvements in user data caching and aggregation algorithms that significantly improve I/O performance. A new decomposition strategy has been introduced which gives the user more ability to tune I/O performance.

This user's guide provides information on how to use the SCORPIO library and an overview of the internal algorithms used by the library.

• Introduction
• Installing SCORPIO
• Install Walk-through
• Describing decompositions
• Error Handling
• Testing
• Examples
• Frequently Asked Questions
• SCORPIO user interface
• Guide for Contributors

Acknowledgement

The SCORPIO library can be acknowledged in publications by referring to the research publication below,

@INPROCEEDINGS{11105984,
  author={Krishna, Jayesh and Wu, Danqing and Jacob, Robert and Ganyushin, Dmitry},
  booktitle={2025 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)}, 
  title={SCORPIO: A Parallel I/O library for Exascale Earth System Models}, 
  year={2025},
  volume={},
  number={},
  pages={712-721},
  keywords={Earth;Runtime;File systems;Computational modeling;High performance computing;Computer architecture;Production;Writing;Throughput;Libraries;Parallel I/O;High-performance I/O;Earth system modeling;High-performance computing},
  doi={10.1109/IPDPSW66978.2025.00112}}

The DOI for the SCORPIO library is also given below for reference,

@misc{ doecode_36752,
title = {SCORPIO},
author = {Krishna, Jayesh and Wu, Danqing and Kurc, Tahsin and Edwards, Jim and Hartnett, Edward},
abstractNote = {The SCORPIO (Software for Caching Output and Reads for Parallel I/O) library provides a portable application programming interface (API) for describing, reading and storing array-oriented data that is distributed in memory across multiple processes. It is used by all the model components in DOE's Energy Exascale Earth System Model (E3SM) for reading input data and writing model output. The library also supports rearranging data distributed across multiple processes before using low-level I/O libraries like PnetCDF, NetCDF, and ADIOS to write the data to the file system. The library was derived from the Parallel I/O library.},
}

License

The SCORPIO library is available under a BSD 3-clause license. Please see LICENSE.txt for details