Overview of the Workflow

The Polaris workflow for updating shared Spack environments typically follows this progression:

  1. Updating Conda Dependencies

  2. Updating Spack Dependencies

  3. Deployment and Testing

  4. Adding a New Machine

  5. Deploying the New Versione

  6. Maintaining Past Versions

We begin with some background information, then each of these steps is detailed in its own page. See below for a high-level summary.

Backgraound: How Conda and Spack Work Together in Polaris

Why does Polaris use both Conda and Spack? What roles do they each serve? Before you start, it’s critical to understand how these two systems work together.

πŸ”— Read more

1. Updating Conda Dependencies

Frequently, developers need to updating Conda dependencies at the same time that they are updating shared Spack environments. In such cases, follow the same process you would if you were just updating Conda dependencies but there is no need to bump the alpha version or make a separate PR for those changes.

πŸ”— Read more

2. Updating Spack Dependencies

Updates to shared Spack environments typcially occur when Polaris needs to support new Spack dependencies (e.g. a new MPI-base library or tool) or when new versions of existing Spack dependencies are required. Sometimes, new shared Spack environments are required because system modules have changed (particularly if modules currently used by Polaris are no longer available). This step documents how to propose new Spack dependencies or changes to existing ones.

πŸ”— Read more

4. Deploying and Testing on HPCs

Before full deployment, new versions of Polaris are installed on a subset of HPC platforms for iterative testing and validation. This stage often requires updating updating mache to support new systems or changes in machine configurations, adding package versions to E3SM’s Spack fork, and troubleshooting deployment scripts.

Testing includes everything from basic imports to full workflow runs.

πŸ”— Read more

5. Adding a New Machine

Most of the work for adding a new machine takes place in mache. Here we provide notes on adding new HPCs that are specific to Polaris.

πŸ”— Read more

6. Deploying the Shared Spack Environments

Once test deployments have been made and the rquired test suites are passing:

  • Deploy across all supported HPC machines

  • Merge the version-update PR

  • Announce the release to the community

πŸ”— Read more

7. Maintaining Past Versions

Older versions of Polaris sometimes require maintenance (repairs or deletion).

πŸ”— Read more