Testing zppy

Unit tests

Run all unit tests by doing the following:

pip install . # Install your changes (`python -m pip install .` also works)
python -u -m unittest tests/test_*.py # Run all unit tests

Integration tests

Integration tests must be run on an LCRC machine. Run all integration tests by doing the following:

pip install . # Install your changes (`python -m pip install .` also works)

# Run steps in the section below ("Commands to run before running integration tests")
python -u -m unittest tests/integration/test_*.py # Run all integration tests

Commands to run before running integration tests

Before running tests/integration/test_complete_run.py run the following:

# You'll need to set some paths to your own directories:
# 1) Edit the paths below
# 2) Edit `output` and `www` parameters in `tests/integration/test_complete_run.cfg`
# 3) Edit `actual_images_dir` in `tests/integration/test_complete_run.py`
rm -rf /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_complete_run_www/v2.LR.historical_0201
rm -rf /lcrc/group/e3sm/$USER/zppy_test_complete_run_output/v2.LR.historical_0201/post
zppy -c tests/integration/test_complete_run.cfg

Before running tests/integration/test_bundles.py run the following:

rm -rf /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_bundles_www/v2.LR.historical_0201
rm -rf /lcrc/group/e3sm/$USER/zppy_test_bundles_output/v2.LR.historical_0201/post
zppy -c tests/integration/test_bundles.cfg
# bundle1 and bundle2 should run. After they finish, invoke zppy again to resolve remaining dependencies:
zppy -c tests/integration/test_bundles.cfg
# bundle3 and ilamb should run

Commands to run to replace outdated expected files

To replace the expected files for test_bash_generation.py run the following:

rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bash_files
cd <top level of zppy repo>
# Your output will now become the new expectation.
# You can just move (i.e., not copy) the output since re-running this test will re-generate the output.
mv test_bash_generation_output/post/scripts /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bash_files
# Rerun test
python -u -m unittest tests/integration/test_bash_generation.py

To replace the expected files for test_campaign.py run the following:

cd <top level of zppy repo>
./tests/integration/update_campaign_expected_files.sh

To replace the expected files for test_defaults.py run the following:

rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files
mkdir -p /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files
# Your output will now become the new expectation.
# You can just move (i.e., not copy) the output since re-running this test will re-generate the output.
mv test_defaults_output/post/scripts/*.settings /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files
# Rerun test
python -u -m unittest tests/integration/test_defaults.py

To replace the expected images for test_complete_run.py run the following:

rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run
# Your output will now become the new expectation.
# Copy output so you don't have to rerun zppy to generate the output.
cp -r /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_complete_run_www/v2.LR.historical_0201 /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run
cd /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run
# This file will list all the expected images.
find . -type f -name '*.png' > ../image_list_expected_complete_run.txt
cd <top level of zppy repo>
# Rerun test
python -u -m unittest tests/integration/test_complete_run.py

To replace the expected images for test_bundles.py run the following:

rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles
# Your output will now become the new expectation.
# Copy output so you don't have to rerun zppy to generate the output.
cp -r /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_bundles_www/v2.LR.historical_0201 /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles
mkdir -p /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles/bundle_files
cp -r /lcrc/group/e3sm/$USER/zppy_test_bundles_output/v2.LR.historical_0201/post/scripts/bundle*.bash /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles/bundle_files
cd /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles
# This file will list all the expected images.
find . -type f -name '*.png' > ../image_list_expected_bundles.txt
cd <top level of zppy repo>
# Rerun test
python -u -m unittest tests/integration/test_bundles.py

Automated tests

We have a GitHub Actions Continuous Integration / Continuous Delivery (CI/CD) workflow.

The unit tests are run automatically as part of this. As mentioned earlier, integration tests must be run on an LCRC machine.