Testing zppy¶
Unit tests¶
Run all unit tests by doing the following:
pip install . # Install your changes (`python -m pip install .` also works) python -u -m unittest tests/test_*.py # Run all unit tests
Integration tests¶
Integration tests must be run on an LCRC machine. Run all integration tests by doing the following:
pip install . # Install your changes (`python -m pip install .` also works) # Run steps in the section below ("Commands to run before running integration tests") python -u -m unittest tests/integration/test_*.py # Run all integration tests
Commands to run before running integration tests¶
Before running tests/integration/test_complete_run.py
run the following:
# You'll need to set some paths to your own directories: # 1) Edit the paths below # 2) Edit `output` and `www` parameters in `tests/integration/test_complete_run.cfg` # 3) Edit `actual_images_dir` in `tests/integration/test_complete_run.py` rm -rf /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_complete_run_www/v2.LR.historical_0201 rm -rf /lcrc/group/e3sm/$USER/zppy_test_complete_run_output/v2.LR.historical_0201/post zppy -c tests/integration/test_complete_run.cfg
Before running tests/integration/test_bundles.py
run the following:
rm -rf /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_bundles_www/v2.LR.historical_0201 rm -rf /lcrc/group/e3sm/$USER/zppy_test_bundles_output/v2.LR.historical_0201/post zppy -c tests/integration/test_bundles.cfg # bundle1 and bundle2 should run. After they finish, invoke zppy again to resolve remaining dependencies: zppy -c tests/integration/test_bundles.cfg # bundle3 and ilamb should run
Commands to run to replace outdated expected files¶
To replace the expected files for test_bash_generation.py
run the following:
rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bash_files cd <top level of zppy repo> # Your output will now become the new expectation. # You can just move (i.e., not copy) the output since re-running this test will re-generate the output. mv test_bash_generation_output/post/scripts /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bash_files # Rerun test python -u -m unittest tests/integration/test_bash_generation.py
To replace the expected files for test_campaign.py
run the following:
cd <top level of zppy repo> ./tests/integration/update_campaign_expected_files.sh
To replace the expected files for test_defaults.py
run the following:
rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files mkdir -p /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files # Your output will now become the new expectation. # You can just move (i.e., not copy) the output since re-running this test will re-generate the output. mv test_defaults_output/post/scripts/*.settings /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files # Rerun test python -u -m unittest tests/integration/test_defaults.py
To replace the expected images for test_complete_run.py
run the following:
rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run # Your output will now become the new expectation. # Copy output so you don't have to rerun zppy to generate the output. cp -r /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_complete_run_www/v2.LR.historical_0201 /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run cd /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run # This file will list all the expected images. find . -type f -name '*.png' > ../image_list_expected_complete_run.txt cd <top level of zppy repo> # Rerun test python -u -m unittest tests/integration/test_complete_run.py
To replace the expected images for test_bundles.py
run the following:
rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles # Your output will now become the new expectation. # Copy output so you don't have to rerun zppy to generate the output. cp -r /lcrc/group/e3sm/public_html/diagnostic_output/$USER/zppy_test_bundles_www/v2.LR.historical_0201 /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles mkdir -p /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles/bundle_files cp -r /lcrc/group/e3sm/$USER/zppy_test_bundles_output/v2.LR.historical_0201/post/scripts/bundle*.bash /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles/bundle_files cd /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bundles # This file will list all the expected images. find . -type f -name '*.png' > ../image_list_expected_bundles.txt cd <top level of zppy repo> # Rerun test python -u -m unittest tests/integration/test_bundles.py
Automated tests¶
We have a GitHub Actions Continuous Integration / Continuous Delivery (CI/CD) workflow.
The unit tests are run automatically as part of this. As mentioned earlier, integration tests must be run on an LCRC machine.