The SCORPIO code is available on github at https://github.com/E3SM-Project/scorpio. For questions about downloading or developing this code please email jayes.nosp@m.h@an.nosp@m.l.gov. Spack package for the SCORPIO library is available here.

Dependencies

SCORPIO uses CMake for configuring the library. The library requires a C, C++ and Fortran compiler and an MPI library.

SCORPIO can use NetCDF (version 4.3.3+), PnetCDF (version 1.6.1+), HDF5 or ADIOS2 for I/O. To use parallel I/O with NetCDF the NetCDF library should be built with MPI, which requires that it be linked with an MPI-enabled version of HDF5. Optionally, NetCDF can be built with DAP support, which introduces a dependency on CURL. Additionally, HDF5 introduces dependencies on LIBZ and (optionally) SZIP libraries. I/O libraries like HDF5 and ADIOS2 support data compression using other libraries like BLOSC2 (lossless data compression) and ZFP (lossy data compression). SCORPIO supports data compression using NetCDF, HDF5 and ADIOS2 I/O libraries.

Configuring with CMake

To configure the build, SCORPIO requires CMake version 3.7+. The typical configuration with CMake can be done as follows,

  > CC=mpicc CXX=mpicxx FC=mpif90 cmake [-DOPTION1=value1 -DOPTION2=value2 ...] /path/to/scorpio/source

where mpicc, mpicxx and mpif90 are the appropriate MPI-enabled compiler wrappers for your system.

The OPTIONS section typically should consist of pointers to the install locations for various dependencies, assuming these dependencies are not located in canonical search locations.

For each dependency XXX, one can specify the location of its installation path with the CMake variable XXX_PATH. You can specify the install directory for the different dependencies using the CMake configuration options as shown below

CC=mpicc CXX=mpicxx FC=mpif90 cmake -DNetCDF_C_PATH=/path/to/netcdf-c \
               -DNetCDF_Fortran_PATH=/path/to/netcdf-fortran \
               -DPnetCDF_PATH=/path/to/pnetcdf \
               -DHDF5_PATH=/path/to/hdf5 \
               /path/to/scorpio/source

This works for the dependencies: NetCDF, PnetCDF, HDF5, LIBZ, SZIP. For adding support for ADIOS2, set the environment variable ADIOS2_DIR to point to the installation directory.

For specific instructions to install on various commonly used super computers, please read the walk-through guide to SCORPIO Installation.

CMake configuration options

CMake configuration options are specified in the "-DCONFIG_OPTION1=CONFIG_OPTION1_VAL" format

> cmake [-DCONFIG_OPTION1=CONFIG_OPTION1_VALUE1 -DCONFIG_OPTION2=CONFIG_OPTION2_VALUE2 ...] /path/to/scorpio/source

Common CMake configuration options

PIO_ENABLE_FORTRAN: Enable the Fortran library interface (default ON)
PIO_ENABLE_TIMING: Collect timing statistics using the GPTL timing library. The application needs to initialize/finalize the GPTL library (default ON)
PIO_ENABLE_INTERNAL_TIMING: Gather and ouput timing statistics using GPTL within the library. The timing statistics is available in "piorwinfo*.dat". The library initializes/finalizes the GPTL library (default OFF)
PIO_ENABLE_API_TRACING: Enable tracing of SCORPIO APIsi (default OFF)
PIO_ENABLE_IO_STATS: Enable SCORPIO I/O performance statistics. The I/O statistics is available in "io_perf_summary*"(default ON)
PIO_ENABLE_LOGGING: Enable debug logging. The log files, "pio_log*", are created per MPI process (and these files can be large) (default OFF)
PIO_ENABLE_DOC: Build SCORPIO documentation (default ON)
PIO_ENABLE_COVERAGE: Enable code coverage using gcov (default OFF)
PIO_ENABLE_TOOLS: Build SCORPIO utilities (convert between file formats, get file information etc) (default ON)
PIO_ENABLE_EXAMPLES: Build SCORPIO examples available in SCORPIO_SOURCE/examples directory (default OFF)
PIO_INTERNAL_DOC: BuildSCORPIO developer documentation (default OFF)
PIO_USE_MPISERIAL: Build SCORPIO with the mpi-serial library instead of an MPI library (default OFF)
PIO_USE_MALLOC: Use native malloc (instead of the BGET libarry) for memory allocations (default OFF)
PIO_USE_INDEP_MODE: Enable PnetCDF independent data mode for non-distributed data, allowing MPI processes to write data independently (default ON)
WITH_PNETCDF: Require the use of the PnetCDF library (default ON)
WITH_NETCDF: Require the use of the NetCDF library (default ON)
PIO_ENABLE_NCZARR: Enable support for NCZarr in the NetCDF library (default OFF)
WITH_ADIOS2: Require the use of the ADIOS2 library (default OFF)
WITH_HDF5: Require the use of the HDF5 library (default OFF)

CMake Configuration Options for data compression

ADIOS_USE_COMPRESSION: Enable data compression using ADIOS. By default lossless data compression is used. (default OFF)
ADIOS_USE_LOSSY_COMPRESSION: Enable lossy data compression using ADIOS (default OFF)
HDF5_USE_COMPRESSION: Enable data compression using HDF5. By default lossless data compression is used. (default OFF)
HDF5_USE_LOSSY_COMPRESSION: Enable lossy data compression using HDF5 (default OFF)

CMake Configuration Options related to Testing

ADIOS_BP2NC_TEST: Enable testing of conversion of the ADIOS2 BP file format to the NetCDF format (default OFF)
PIO_ENABLE_TESTS: Configure/Enable the SCORPIO tests (default OFF)
PIO_ENABLE_LARGE_TESTS: Configure/Enable SCORPIO tests that write a large amount of data (default OFF)
PIO_VALGRIND_CHECK: Enable memory leak check using valgrind (default OFF)

Advanced CMake Configuration Options

PIO_ENABLE_API_VAR_TRACING: Enable tracing of variables when tracing SCORPIO APIs (default OFF)
PIO_USE_FORTRAN_LEGACY_LIB: Use the legacy Fortran library interface (for backward compatibility with old sources, default OFF)
PIO_MICRO_TIMING: Enable internal micro timers (default OFF)
PIO_SAVE_DECOMPS: Output the I/O decomposition information (default OFF)
PIO_LIMIT_CACHED_IO_REGIONS: Limit the number of non-contiguous regions cached in an IO process. If enabled, by default the library caches 64K I/O regions (default OFF)
PIO_MAX_CACHED_IO_REGIONS: The number of non-contiguous regions cached in an I/O process (default UNSET)
PIO_MAX_LUSTRE_OSTS: The number of Lustre OSTs to use for I/O (This value is set by default for LCF machines)
PIO_STRIPING_UNIT: The Lustre striping unit for I/O (Tihs value is set by default for LCF machines)
PIO_RESERVED_FILE_HEADER_SIZE: The size (in bytes) reserved for the NetCDF header (default 10KB)
PIO_CHUNK_SIZE: The size of each chunk of output data for NetCDF and HDF5 files (default 4MB)
PIO_MAX_ADIOS_DECOMPS: The maximum number of I/O decompositions registered with ADIOS (default 64K)
PIO_MAX_CACHED_STEPS_FOR_ADIOS: The maximum number of ADIOS steps of data cached before syncing the data to file (default 128)
PIO_REARR_ANY_SUBSET_RANGE: The range of lengths (max across all procs) of the local decomposition map for the ANY rearranger before switching from BOX to SUBSET (default : (512K, inf) )
ADIOS_NO_DECOMPS: Save no I/O decomposition data to ADIOS BP files. The I/O decomposition information is required for converting ADIOS2 BP files output to NetCDF (default OFF)
SPIO_HDF5_FLUSH_AFTER_COLL_WR: Flush file buffers after each collective write using HDF5 (default OFF)
SPIO_OVERRIDE_ADIOS_WITH_PNETCDF_FNAME_REGEX: Regular expression for filenames that need to use PnetCDF instead of the default ADIOS I/O type
SPIO_OVERRIDE_HDF5_COMPRESSION_VNAME_REGEX: Regular expression for variable names that need to override (disable) the default compression options
SPIO_COMPRESSION_OPTIONS: Data compression settings to override defaults (default UNSET). e.g. cmake -DSPIO_COMPRESSION_OPTIONS="COMPRESSION_LIBRARY=ADIOS2:Blosc2:Zstd; SPIO_ADIOS2_BLOSC2_COMPRESSION_LEVEL=2; SPIO_ADIOS2_BLOSC2_SHUFFLE_METHOD=BLOSC_BITSHUFFLE; COMPRESSION_LIBRARY=HDF5:Blosc2:Zstd; SPIO_HDF5_BLOSC2_COMPRESSION_LEVEL=2; SPIO_HDF5_BLOSC2_SHUFFLE_METHOD=BLOSC_BITSHUFFLE;" ...

Building

After configuring SCORPIO using CMake as mentioned above the library can be built using the make command

    > make

Testing

The tests can be be built using the tests target.

    > make tests

Once the tests have been built, you may run tests using the CMake test driver, ctest

    > ctest

Alternatively, you may build the test executables and then run tests using the check target

    > make check

The tests can be run from the build directory on the command line on machines that do not use a batch job scheduler. For machines that use a batch job scheduler these commands need to be run from a batch job. Please refer to the appropriate user guides for information on how to create run executables from a batch job.

Installing

The install directory is specified using the CMAKE_INSTALL_PREFIX CMake configuration option. To install the library use the install target

    > make install

Examples

To build the examples use the examples target

    > make examples