Project Standards¶

Development Environment¶

If you haven’t already, please visit “(b) Development Environment” to set up your local development environment.

Version Control¶

The repository uses a fork-based Git workflow with tag releases.

Guidelines¶

master must always be deployable
All changes are made through support branches on forks
Rebase with master to avoid/resolve conflicts
Make sure pre-commit checks pass when committing (enforced in CI/CD build)
Open a pull-request (PR) early for discussion
Once the CI/CD build passes and PR is approved, squash and rebase your commits
Merge PR into master and delete the branch

Things to Avoid¶

Don’t merge in broken or commented out code
Don’t commit onto master directly
Don’t merge with conflicts (handle conflicts upon rebasing)

Source: https://gist.github.com/jbenet/ee6c9ac48068889b0912

Pre-commit¶

The repository uses the pre-commit package to manage pre-commit hooks. These hooks help enforce quality assurance standards and identify simple issues at the commit level before submitting code reviews.

pre-commit Flow¶

Helpful Commands¶

Install into your cloned repo

                            conda activate e3sm_diags_env_dev
pre-commit install

Automatically run all pre-commit hooks (just commit)

                            # Tip: If there is an issue with pre-commit, you can bypass with the `--no-verify` flag. Please do NOT use this on a regular basis.
git commit -m '...'

pre-commit Output¶

Manually run all pre-commit hooks

                            pre-commit run --all-files

                          

Run individual hook

                            # Available hook ids: trailing-whitespace, end-of-file-fixer, check-yaml, black, isort, flake8, mypy
pre-commit run <hook_id>

Squash and Rebase Commits¶

Before you merge a support branch back into master, the branch is typically squashed down to a single* buildable commit, and then rebased on top of the main repo’s master branch.

* In some cases, it might be logical to have multiple squashed commits, as long as each commit passes the CI/CD build

Why squash and rebase commits?

Ensures build passes from the commit
Cleans up Git history for easy navigation
Makes collaboration and review process more efficient
Makes handling conflicts from rebasing simple since you only have to deal with conflicted commits
Makes git bisect easier and more effective to use. For example, it will show the exact commit that introduced a bug since the commit contains a relatively small changeset. On the otherhand, merge commits make it much harder since it includes a large changeset.

How to squash and rebase commits¶

Assuming that you followed “(b) Development Environment”:

Sync master with the main repo’s master

                                git checkout master
git rebase <upstream-origin>/master
git push -f <fork-origin> master

                              

Get the SHA of the commit OR number of commits to rebase to
```
git log --graph --decorate --pretty=oneline --abbrev-commit
```

Squash commits:

                                git rebase -i [SHA]

# OR

git rebase -i HEAD~[NUMBER OF COMMITS]

Rebase branch onto master

                                git checkout <branch-name>
git rebase master
git push -f <fork-origin> <branch-name>

                              

Make sure your squashed commit messages are refined

Force push to remote branch

                                git push -f <fork-origin> <branch-name>

                              

Source: https://blog.carbonfive.com/always-squash-and-rebase-your-git-commits/

Code Quality Assurance¶

This project uses several tools for code formatting, linting, and type checking listed below.

Code Formatting: black
Linting: flake8, isort
Optional Type Checking: mypy

You can run them as hooks manually/automatically when committing using pre-commit, or manually through the terminal or IDE/text editor.

Helpful Commands¶

Run a tool

                              # Available tool names: black, flake8, isort, mypy
<tool_name> .

Continuous Integration / Continuous Delivery (CI/CD)¶

This project uses GitHub Actions to run two CI/CD workflows.

CI/CD Build Workflow

This workflow is triggered by Git pull_request and push (merging PRs) events to the the main repo’s master.

Jobs:

Run pre-commit for formatting, linting, and type checking

Run test suite in a conda environment

Publish latest master docs (only on push)

CI/CD Release Workflow

This workflow is triggered by the Git publish event, which occurs when a new release is tagged.

Jobs:

Publish new release docs

Publish Anaconda package

Style Guide¶

This project follows the Black code style. Please read about it more here.

API Documentation¶

In most cases, code should be self-documenting.

If necessary, documentation should explain why something is done, its purpose, and its goal. The code shows how it is done, so commenting on this can be redundant.

Guidelines¶

Embrace documentation as an integral part of the overall development process
Treat documenting as code and follow principles such as Don’t Repeat Yourself and Easier to Change
Use comments and docstrings to explain ambigiuity, complexity, or to avoid confusion
Co-locate API documentation with related code
Use Python type annotations and type comments where helpful

Things to Avoid¶

Don’t write comments as a crutch for poor code
Don’t comment every function, data structure, type declaration