Software Quality Assurance Tooling - Wintersession 2024

Software Quality Assurance Tooling
After today you will know SQAT
Henry Schreiner
Last update: Jan 23, 2024
ISciNumPy
https://iscinumpy.dev
Scienti
fi
c-Python Development Guide
https://learn.scienti
fi
c-python.org/development
https://github.com/henryiii/sqat-example

Language
We’ll focus on Python (especially at
fi
rst)
But the general concepts are around in most other languages
You just need to
fi
nd the matching tool(s)

Packaging aside: pipx
3
$ pip install <application>
$ <application>
I’m sure you’ve seen this: Examples of applications:
build: make SDists and wheels
twine: upload SDists and wheels
cibuildwheel: make redistributable wheels
nox/tox: Python task runners
jupylite: WebAssembly Python site builder
ruff: Python code linter and formatter
pypi-command-line: query PyPI
uproot-browser: ROOT file browser (HEP)
tiptop: fancy top-style monitor
rich-cli: pretty print files
cookiecutter: template packages
clang-format: format C/C++/CUDA code
pre-commit: general CQA tool
cmake: build system generator
meson: another build system generator
ninja: build system

3
$ <application>
ninja: build system
Packages can con
fl
ict
Updates get slower over time
Lose track of why things are installed
Manual updates are painful
Hates Python being replaced
Solution: venvs aren’t great for apps!

3
$ <application>
ninja: build system
Packages can con
fl
ict
$ pipx install <application>
$ <application>
Better!
Automatic venv for each package
No con
fl
icts ever
Everything updatable / replaceable
Doesn’t like Python being replaced

3
$ <application>
ninja: build system
Packages can con
fl
ict
$ pipx install <application>
$ <application>
Better!
Automatic venv for each package
No con
fl
icts ever
Everything updatable / replaceable
Doesn’t like Python being replaced
$ pipx run <application>
Best!
Automatic venv caching
Never more than a week old
No pre-install or setup
No maintenance
Replace Python at will
pipx run --spec git+https://github.com/henryiii/rich-cli@patch-1 rich
pipx has
fi
rst
class support
on GHA & Azure!
manylinux too

Quick scripts solution!
5
# /// script
# requirements = ["rich"]
# ///
import rich
rich.print("[blue]This worked!")
pipx run ./print_blue.py
Requires pipx 1.4.2+, or an upcoming release of Hatch

Task runner aside: Nox
6
Make
fi
les
Custom language
Painful to write
Painful to maintain
Looks like garbage
OS dependent
No Python environments
Everywhere
Tox
Custom language
Concise to write
Tricky to read
Ties you to tox
OS independent
Python environments
Python package
Nox
Python, mimics pytest
Simple but verbose
Easy to read
Teaches commands
OS independent
Python environments
Python package
Other task runners available for other purposes, like Rake (Ruby)
Hatch
TOML con
fi
g
Intermediate
Intermediate
Integrated with packaging
OS independent
Python environments
Python package

Writing a nox
fi
le.py
7
import nox
@nox.session(python=["3.8", "3.9", "3.10", "3.11", "3.12"])
def tests(session: nox.Session) -> None:
"""
Run the unit and regular tests.
"""
session.install("-e.[test]")
session.run("pytest", *session.posargs)

Running nox
8
~/g/s/uproot-browser   henryiii/feat/logo *$  nox -s tests-3.9
nox > Running session tests-3.9
nox > Creating virtual environment (virtualenv) using python3.9 in .nox/tests-3-9
nox > python -m pip install '.[test]'
nox > pytest
=========================================== test session starts ===========================================
platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/henryschreiner/git/scikit-hep/uproot-browser, configfile: pyproject.toml, testpaths: tests
collected 3 items
tests/test_dirs.py .. [ 66%]
tests/test_package.py . [100%]
=========================================== 3 passed in 0.01s =============================================
nox > Session tests-3.9 was successful.

Features of nox
9
Full control over environments
Easy
fl
y-by contributions
Transparent, simple .nox directory
Conda support
Trade speed for reproducibility
Some ideas for sessions
lint
tests
docs
build
bump
pylint
regenerate
update_pins
check_manifest
make_changelog
update_python_dependencies
See
pypa/cibuildwheel
pypa/manylinux
scikit-hep/hist
scikit-hep/boost-histogram
pybind/pybind11
scientific-python/cookie
scientific-python/repo-review
scikit-hep/scikit-hep.github.io
Optional environment reuse
Use -R for speed! (Reuse environment and skip installs)

Python launcher for Unix
10
Rust implementation of “py” for UNIX
But also automatically picks up .venv folder!
Meant for lazy experts
Launcher
$ py -m pytest
Classic
$ . .venv/bin/activate
(.venv) $ python -m pytest
(.venv) $ deactivate
Classic, take 2
$ .venv/bin/python -m pytest

Cookiecutter
11
Quickly set up a project
Takes options
Scienti
fi
c-python/cookie is a great cookiecutter for Python!
How to run
pipx run cookiecutter gh:scientific-python/cookie

Code Quality
13
Why does code quality matter?
Improve readability
Find errors before they happen
Avoid historical baggage
Reduce merge con
fl
icts
Warm fuzzy feelings
How to run
Discussion of checks
(Opinionated)
Mostly focusing on Python today

pre-commit
14
Poorly named?
Has a pre-commit hook mode
You don’t have to use it that way!
Generic check runner
conda
coursier
dart
docker
docker_image
dotnet
fail
golang
lua
node
perl
python
python_venv
r
ruby
rust
swift
pygrep
script
system
Written in Python
pipx, nox, homebrew, etc.
Designed for speed & reproducibility
Ultra fast environment caching
Locked environments
Easy autoupdate command
pre-commit.ci
Automatic updates
Automatic
fi
xes for PRs
Large library of hooks
https://pre-commit.com/hooks.html
Custom hooks are simple

Con
fi
guring pre-commit
15
Design
A hook is just a YAML dict
Fields can be overridden
Environments globally cached by git tag
Supports checks and
fi
xers
You can have as many as you want
Must use a static tag
# .pre-commit-config.yaml
hooks:
- repo: https://github.com/abravalheri/validate-pyproject
rev: "0.15"
hooks:
- id: validate-pyproject You write this

Con
fi
guring pre-commit
15
Design
A hook is just a YAML dict
Fields can be overridden
Environments globally cached by git tag
Supports checks and
fi
xers
You can have as many as you want
Must use a static tag
# .pre-commit-config.yaml
hooks:
rev: "0.15"
hooks:
- id: validate-pyproject
# validate-pyproject .pre-commit-hooks.yaml
name: Validate pyproject.toml
description: >
Validation library for a simple check
on pyproject.toml, including optional dependencies
language: python
files: ^pyproject.toml$
entry: validate-pyproject
additional_dependencies:
- .[all]
You write this
Formatter author writes this

Options for pre-commit
16
Selected options
fi
les: explicit include regex
exclude: explicit exclude regex
types_or/types/exclude_types:
fi
le types
args: control arguments
additional_dependencies: extra things to install
stages: select the git stage (like manual)

Running pre-commit
17
Run all checks
pre-commit run -a
Update all hooks
pre-commit autoupdate

Running pre-commit
17
Run all checks
pre-commit run -a
Update all hooks
Install as a pre-commit hook
pre-commit install
(Skip with git commit -n)

Running pre-commit
17
Run all checks
pre-commit run -a
Update all hooks
Install as a pre-commit hook
pre-commit install
(Skip with git commit -n)
Skip checks
SKIP=… <run>
Run one check
pre-commit run -a <id>
Run manual stage
pre-commit run --hook-stage manual

Examples of pre-commit checks
18
Almost everything following in this talk
- repo: local
hooks:
- id: disallow-caps
name: Disallow improper capitalization
language: pygrep
entry: PyBind|Numpy|Cmake|CCache|Github|PyTest
exclude: .pre-commit-config.yaml
Don’t grep the
fi
le this is in!
“Entry” is the grep, in this case
Using pygrep “language”
Custom hook

pre-commit/pre-commit-hooks
19
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: "v4.5.0"
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-merge-conflict
- id: check-symlinks
- id: check-yaml
- id: debug-statements
- id: end-of-file-fixer
- id: mixed-line-ending
- id: requirements-txt-fixer
- id: trailing-whitespace
Small common checks
Some Python leaning
Some pre-commit hook specialization

pre-commit/pygrep-hooks
20
Small common pygreps
- repo: https://github.com/pre-commit/pygrep-hooks
rev: "v1.10.0"
hooks:
- id: rst-backticks
- id: rst-directive-colons
- id: rst-inline-touching-normal

CI (GitHub Actions)
21
on:
pull_request:
push:
branches:
- main
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: "3.x"
- uses: pre-commit/action@v3.0.0
Great, fast caching, but maintenance only - replaced by pre-commit.ci

CI (GitHub Actions)
21
on:
pull_request:
push:
branches:
- main
jobs:
lint:
steps:
- uses: actions/setup-python@v4
with:
python-version: "3.x"
- uses: pre-commit/action@v3.0.0
Great, fast caching, but maintenance only - replaced by pre-commit.ci
on:
pull_request:
push:
branches:
- main
jobs:
lint:
steps:
- run: pipx run nox -s lint
@nox.session
def lint(session: nox.Session) -> None:
session.install("pre-commit")
session.run("pre-commit", "run", "--all-files", "--show-diff-on-failure", *session.posargs)

Useful GitHub Actions
22
actions/checkout
actions/setup-python
actions/cache
actions/upload-artifact
actions/download-artifact
ilammy/msvc-dev-cmd
jwlawson/actions-setup-cmake
excitedleigh/setup-nox
pypa/gh-action-pypi-publish
pre-commit/action
conda-incubator/setup-miniconda
peaceiris/actions-gh-pages
ruby/setup-miniconda
Writing your own composite action is really easy!

Part 1: One tool to rule them all
23

Ruff
24
A new entry in the Python linting/formatting space, amazing adoption in a year(ish)
100x faster than existing Python linters
Has support for
fi
xers!
Implements all of (modern)
fl
ake8’s checks
Implements dozens of
fl
ake8 plugins
Fixes many long-standing issues in plugins
Over 700 rules (!!!)
0 dependencies
Con
fi
gured with pyproject.toml
Has a Black-like formatter too, 30x faster than Black!
Only binary platforms (Rust compiled)
Doesn’t support user plugins
Online version
https://play.ru
ff
.rs
0s 20s 40s 60s
Ruff
Autoflake
Flake8
Pyflakes
Pycodestyle
Pylint
0.29s
6.18s
12.26s
15.79s
46.92s
> 60s

Ruff
24
A new entry in the Python linting/formatting space, amazing adoption in a year(ish)
100x faster than existing Python linters
Has support for
fi
xers!
Implements all of (modern)
fl
ake8’s checks
Implements dozens of
fl
ake8 plugins
Fixes many long-standing issues in plugins
Over 700 rules (!!!)
0 dependencies
Con
fi
gured with pyproject.toml
Has a Black-like formatter too, 30x faster than Black!
Only binary platforms (Rust compiled)
Doesn’t support user plugins
Online version
https://play.ru
ff
.rs

Ruff con
fi
g example
25
[tool.ruff]
src = ["src"]
[tool.ruff.lint]
extend-select = [
"B", # flake8-bugbear
"I", # isort
"ARG", # flake8-unused-arguments
"C4", # flake8-comprehensions
"EM", # flake8-errmsg
"ICN", # flake8-import-conventions
"PGH", # pygrep-hooks
"PIE", # flake8-pie
"PL", # pylint
"PT", # flake8-pytest-style
"RET", # flake8-return
"RUF", # Ruff-specific
"SIM", # flake8-simplify
"T20", # flake8-print
"UP", # pyupgrade
"YTT", # flake8-2020
]
typing-modules = ["somepackage._compat.typing"]
[tool.ruff.lint.per-file-ignores]
"tests/**" = ["T20"]
"noxfile.py" = ["T20"]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.1.14"
hooks:
- id: ruff
args: ["--fix", "--show-fixes"]
- id: ruff-format
Flake8 con
fi
g? Try:
pipx run flake8-to-ruff .flake8
fi
c-python.org/development/guides/style/#ru
f

Ruff-format
26
Black
Close to the one true format for Python
Almost not con
fi
gurable (this is a feature)
A good standard is better than perfection
Designed to reduce merge con
fl
icts
Reading blacked code is fast
Write your code to produce nice formatting
You can disable line/lines if you have to
Workaround for single quotes (use double)
Magic trailing comma
Ru
ff
’s formatter
99.9% compatible with Black
A little bit more con
fi
gurable
Fast(er)
Already present if using Ru
f

Write for good format
27
raise RuntimeError(
"This was not a valid value for some_value: {}".format(repr(some_value))
)
Bad:
Ru
ff
can check for this
and rewrite it for you!

Write for good format
27
raise RuntimeError(
"This was not a valid value for some_value: {}".format(repr(some_value))
)
Bad:
msg = f"This was not a valid value for some_value: {some_value!r}"
raise RuntimeError(msg)
Good:
Better stacktrace
More readable
Two lines instead of three
Faster (f-string)
Ru
ff
can check for this
and rewrite it for you!

Using code formatters
28
Existing projects
Apply all-at-once, not spread out over time
Add the format commit to .git-blame-ignore-revs
(GitHub now recognizes this
fi
le, too!)

Running Ruff on notebooks
29
pre-commit? Add:
types_or: [python, pyi, jupyter]
Native support
Linter and formatter support notebooks
Generally have to opt in
md/rst support planned

Formatting code snippets
30
Ru
ff
native support planned soon!
- repo: https://github.com/adamchainz/blacken-docs
rev: "1.16.0"
hooks:
- id: blacken-docs
additional_dependencies: [black==23.*]
Blacken docs
Adapts black to md/rst
fi
les

Ruff linter
31
Groups of rules
Most are based on some existing tool / plugin
Opt in (recommended) or use ALL
--preview enables lots more
Fixing code
--fix --show-fixes on the command line
--unsafe-fixes for even more
fi
xes
Can disable
fi
xes by code
Running Ru
ff
Doesn’t depend on version of Python!
Doesn’t require any environment setup!
Easy to run locally as well as in pre-commit
Can integrate with VSCode or any LSP editor

Using code linters
32
Existing projects
Feel free to build a long ignore list
Work on one or a few at a time
You don’t have to have every check

Default codes
33
F: PyFlakes (default)
Unused modules & variables
String formatting mistakes
No placeholders in f-string
Dictionary key repetition
Assert a tuple (it’s always true)
Various syntax errors
Unde
fi
ned names
Rede
fi
nition of unused var ❤ pytest
C90: McCabe
Complexity checks
E: PyCodeStyle (subset default)
Style checks

Other useful codes
34
B: Bugbear
Do not use bare except
No mutable argument defaults
getattr(x, "const") should be x.const
No assert False, use raise AssertionError
Pointless comparison ❤ pytest
T20:
fl
ake8-print
Avoid leaking debugging print statements
D: pydocstyle
Documentation requirements
PERF: per
fl
int
Detect common expressions with faster idioms
SIM:
fl
ake8-simplify
Simpli
fi
er form for expression
C4:
fl
ake8-comprehensions
Comprehension simpli
fi
cation
PTH:
fl
ake8-use-pathlib
Use pathlib instead of os.path
And many more!

Ruff’s own codes
35
NPY: numpy rules
Can detect 2.0 upgrade changes
RUF codes
Unicode checks
Unused noqa (
fi
xer can remove unused!)
Various assorted checks
See all the codes at:
https://docs.astral.sh/ru
ff
/rules

Code I: isort
36
Sort your Python imports
Very con
fi
gurable
Reduces merge con
fl
icts
Grouping imports helps readers
Can inject future imports
args: ["-a", "from __future__ import annotations"]
Default groupings
Future imports
Stdlib imports
Third party packages
Local imports
from __future__ import annotations
import dataclasses
import graphlib
import textwrap
from collections.abc import Mapping, Set
from typing import Any, TypeVar
import markdown_it
from .checks import Check
from .families import Family, collect_families
from .fixtures import pyproject
from .ghpath import EmptyTraversable

Code UP: pyupgrade
37
Update Python syntax
Avoid deprecated or obsolete code
Fairly cautious
Can target a speci
fi
c Python 3 min
(Mostly) not con
fi
gurable
Remove static if sys.version_info blocks
Python 2.7
Set literals
Dictionary comprehensions
Generators in functions
Format speci
fi
er & .format ⚙
Comparison for const literals (3.8 warn)
Invalid escapes
Python 3
Unicode literals
Long literals, octal literals
Modern super()
New style classes
Future import removal
yield from
Remove six compatibility code
io.open -> open
Remove error aliases
Python 3.x
f-strings (partial) (3.6) ⚙
NamedTuple/TypedDict (3.6)
subprocess.run updates (3.7)
lru_cache parens (3.8)
lru_cache(None) -> cache (3.9)
Typing & annotation rewrites (various)
abspath(__file__) removal (3.9)
Before After
for a, b in c:
yield (a, b)
yield from c
"{foo} {bar}".format(foo=foo, bar=bar) f"{foo} {bar}"
dict([(a, b) for a, b in y]) {a: b for a, b in y}

pyupgrade limits
38
PyUpgrade does not over modernize
isinstance(x, (int, str)) -> isinstance(x, int | str) (3.10)
No match statement conversions (3.10)
Nothing converts to using walrus := (3.8) (probably a good thing!)
Except for a bit of typing
Optional[int] -> int | None (I like this one now, though)
❌

Notebook cleaner
40
hooks:
- repo: https://github.com/kynan/nbstripout
rev: "0.6.1"
hooks:
- id: nbstripout
Remove outputs from notebooks
Best if not stored in VCS
You can render outputs in JupyterBook, etc.
Use Binder or JupyterLite

hooks:
- repo: https://gitlab.com/pycqa/flake8
rev: "7.0.0"
hooks:
- id: flake8
Flake8
41
Fast simple extendable linter
Very con
fi
gurable: setup.cfg or .
fl
ake8
Doesn’t support pyproject.toml
Many plugins, local plugins easy
No auto-
fi
xers like rubocop (Ruby)
Still great for custom checks
# .flake8
[flake8]
max-complexity = 12
extend-ignore = E203, E501, E722, B950
extend-select = B9

Custom local
fl
ake8 plugin
42
import ast
import sys
from typing import NamedTuple, Iterator
class Flake8ASTErrorInfo(NamedTuple):
line_number: int
offset: int
msg: str
cls: type # unused

Custom local
fl
ake8 plugin
43
class Visitor(ast.NodeVisitor):
msg = "AK101 exception must be wrapped in ak._v2._util.*error"
def __init__(self) -> None:
self.errors: list[Flake8ASTErrorInfo] = []
def visit_Raise(self, node: ast.Node) -> None:
if isinstance(node.exc, ast.Call):
if isinstance(node.exc.func, ast.Attribute):
if node.exc.func.attr in {"error", "indexerror"}:
return
if node.exc.func.id in {"ImportError"}:
return
self.errors.append(
Flake8ASTErrorInfo(node.lineno, node.col_offset, self.msg, type(self))
)

Custom local
fl
ake8 plugin
44
class AwkwardASTPlugin:
name = "flake8_awkward"
version = "0.0.0"
def __init__(self, tree: ast.AST) -> None:
self._tree = tree
def run(self) -> Iterator[Flake8ASTErrorInfo]:
visitor = Visitor()
visitor.visit(self._tree)
yield from visitor.errors

Custom local
fl
ake8 plugin
45
[flake8:local-plugins]
extension =
AK1 = flake8_awkward:AwkwardASTPlugin
paths =
./dev/
def main(path: str) -> None:
with open(path) as f:
code = f.read()
node = ast.parse(code)
plugin = AwkwardASTPlugin(node)
for err in plugin.run():
print(f"{path}:{err.line_number}:{err.offset} {err.msg}")
if __name__ == "__main__":
for item in sys.argv[1:]:
main(item)

PyLint
46
PyLint recommends having your project installed, so it is not a good pre-commit hook (though you can do it)
It’s also a bit slow, so a good candidate for nox
@nox.session
def pylint(session: nox.Session) -> None:
session.install("-e.")
session.install("pylint")
session.run("pylint", "src", *session.posargs)
# pyproject.toml
[tool.pylint]
master.py-version = "3.8"
master.jobs = "0"
reports.output-format = "colorized"
similarities.ignore-imports = "yes"
messages_control.enable = ["useless-suppression"]
messages_control.disable = [
"design",
"fixme",
"line-too-long",
"wrong-import-position",
]
Code linter
Can be very opinionated
Signal to noise ratio poor
You will need to disable checks - that’s okay!
A bit more advanced / less static than
fl
ake8
But can catch hard to
fi
nd bugs!
For an example of lots of suppressions:
https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/pyproject.toml
Some parts available in Ruff

Example PyLint rules
47
Duplicate code
Finds large repeated code patterns
Attribute de
fi
ned outside init
Only __init__ should de
fi
ne attributes
No self use
Can be @classmethod or @staticmethod
Unnecessary code
Lambdas, comprehensions, etc.
Unreachable code
Finds things that can’t be reached
Consider using in
x in {stuff} vs chaining or’s
Arguments di
ff
er
Subclass should have matching arguments
Consider iterating dictionary
Better use of dictionary iteration
Consider merging isinstance
You can use a tuple in isinstance
Useless else on loop
They are bad enough when useful :)
Consider using enumerate
Avoid temp variables, idiomatic
Global variable not assigned
You should only declare global to assign

Controversial PyLint rules
48
No else after control-
fl
ow
Guard-style only
Can simply complex control
fl
ow
Removes useless indentation
if x:
return x
else:
return None
# Should be:
if x:
return x
return None
# Or:
return x if x else None
# Or:
return x or None
Design
Too many various things
Too few methods
Can just silence “design”

Controversial PyLint rules
48
No else after control-
fl
ow
Guard-style only
Can simply complex control
fl
ow
Removes useless indentation
if x:
return x
else:
return None
# Should be:
if x:
return x
return None
# Or:
return x if x else None
# Or:
return x or None
Design
Too many various things
Too few methods
Can just silence “design”
(I’m on the in-favor side)

Static type checking: MyPy
49
hooks:
- repo: https://gitlab.com/pre-commit/mirrors-mypy
rev: "v1.8.0"
hooks:
- id: mypy
files: src
Like a linter on steroids
Uses Python typing
Enforces correct type annotations
Designed to be iteratively enabled
Should be in a controlled environment (pre-commit or nox)
Always specify args (bad hook defaults)
Almost always need additional_dependencies
Con
fi
gure in pyproject.toml
Pros
Can catch many things tests normally catch, without writing tests
Therefore it can catch things not covered by tests (yet, hopefully)
Code is more readable with types
Sort of works without types initially
Cons
Lots of work to add all types
Typing can be tricky in Python
Active development area for Python

Con
fi
guring MyPy
50
[tool.mypy]
files = "src"
python_version = "3.8"
warn_unused_configs = true
strict = true
[[tool.mypy.overrides]]
module = [ "numpy.*" ]
ignore_missing_imports = true
Start small
Start without strictness
Add a check at a time
Extra libraries
Try adding them to your environment
You can ignore untyped or slow libraries
You can provide stubs for untyped libraries if you want
Tests?
Adding pytest is rather slow
I prefer to avoid tests, or keep them mostly untyped

Typing tricks
51
Protocols
Better than ABCs, great for duck typing
@typing.runtime_checkable
class Duck(Protocol):
def quack() -> str:
...
def f(x: Duck) -> str:
return x.quack()
class MyDuck:
def quack() -> str:
return "quack"
if typing.TYPE_CHECKING:
_: Duck = typing.cast(MyDuck, None)
Type Narrowing
Integral to how mypy works
x: Union[A, B]
if isinstance(x, A):
reveal_type(x) # A
else:
reveal_type(x) # B
Make a typed package
Must include py.typed marker
fi
le
Always use sys.version_info
Better for readers than try/except, and static
Also sys.platform instead of os.name

Future annotations
52
Classic code (3.5+)
from typing import Union, List
def f(x: int) -> List[int]:
return list(range(x))
def g(x: Union[str, int]) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
Modern code (3.7+)
from __future__ import annotations
def f(x: int) -> list[int]:
def g(x: str | int) -> None:
else:
print("int", x)
Ultramodern code (3.10+)
def f(x: int) -> list[int]:
def g(x: str | int) -> None:
else:
print("int", x)
With the future import, you get all the bene
fi
ts of future code in 3.7+ annotations
Typing is already extra code, simpler is better

Clang-format
54
hooks:
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: "v17.0.6"
hooks:
- id: clang-format
types_or: [c++, c, cuda]
C++ and more code formatter
Very con
fi
gurable: .clang-format
fi
le
Opinion: stay close to llvm style
PyPI clang-format wheels, under 2MB
No more issues with mismatched LLVM!

CMake-format
55
hooks:
- repo: https://github.com/cheshirekow/cmake-format-precommit
rev: "v0.6.13"
hooks:
- id: cmake-format
additional_dependencies: [pyyaml]
CMake code formatter
Very con
fi
gurable: .cmake-format.yaml
fi
le
Anything that helps with CMake!

Markdown & YAML with Prettier
56
hooks:
- repo: https://github.com/pre-commit/mirrors-prettier
rev: "v3.1.0"
hooks:
- id: prettier
types_or: [yaml, markdown, html, css, scss, javascript, json]
args: [--prose-wrap=always]
exclude: "^tests"
JavaScript Linter
Lots of formats supported
A few customization points

ShellCheck
57
hooks:
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: "v0.9.0.6"
hooks:
- id: shellcheck
Linter for bash scripts
Can locally disable
Prioritizes correctness over terseness

CodeSpell
58
hooks:
- repo: https://github.com/codespell-project/codespell
rev: "v2.2.6"
hooks:
- id: shellcheck
args: ["-L", "sur,nd"]
Find common misspellings
Inverted spell checker - looks for misspellings
Can con
fi
gure or provide wordlist
Actually can catch bugs!
Pass -w to
fi
x, too

Schemas
59
hooks:
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: "0.27.3"
hooks:
- id: check-readthedocs
- id: check-github-workflows
Can validate common
fi
les
Can get more from SchemaStore
(Over 700
fi
le types supported)
Can write custom schemas too
hooks:
rev: “0.16"
hooks:
Specialized for pyproject.toml
Supports plugins
SchemaStore support just released
Live demo:
https://scienti
fi
c-python.github.io/repo-review/

See how your repo measures!
60
fi
c-python.org/development/guides/repo-review/

pytest tips
62
Spend time learning pytest
Full of amazing things that really make testing fun!
Tests are code too
Or for C++: Catch2 or doctest, etc.
Also maybe learn Hypothesis for pytest
[tool.pytest.ini_options]
minversion = "6.0"
addopts = [
"-ra",
"--showlocals",
"--strict-markers",
"--strict-config",
]
xfail_strict = true
filterwarnings = [
"error",
]
log_cli_level = "info"
testpaths = [
"tests",
]
Use pytest.approx
Even works on numpy arrays
Remember to test for failures
If you expect a failure, test it!
Test your installed package
That’s how users will get it, not from a directory

pytest tips
62
Spend time learning pytest
Full of amazing things that really make testing fun!
Tests are code too
Or for C++: Catch2 or doctest, etc.
Also maybe learn Hypothesis for pytest
[tool.pytest.ini_options]
minversion = "6.0"
addopts = [
"-ra",
"--showlocals",
"--strict-markers",
"--strict-config",
]
xfail_strict = true
filterwarnings = [
"error",
]
log_cli_level = "info"
testpaths = [
"tests",
]
Don’t let warnings slip by!
Makes logging more useful
Strictness is good
Useful summary
Print out locals on errors
Use pytest.approx
Even works on numpy arrays
Remember to test for failures
If you expect a failure, test it!
Test your installed package
That’s how users will get it, not from a directory

pytest Tricks
63
Mock and Monkeypatch
This is how you make tricky tests “unit” tests
Fixtures
This keeps tests simple and scalable
@pytest.fixture(params=["Linux", "Darwin", "Windows"], autouse=True)
def platform_system(request, monkeypatch):
monkeypatch.setattr(platform, "system", lambda _: request.param)
Parametrize
Directly or in a
fi
xture for reuse
Use conftest.py
Fixtures available in same and nested directories

Running pytest
64
Show locals on failure
--showlocals/-l
Jump into a debugger on failure
--pdb
Start with last failing test
--lf
Jump into a debugger immediately
--trace or use breakpoint()
Run matching tests
-k <expression>
Run speci
fi
c test
filename.py::testname
Run speci
fi
c marker
-m <marker>
Control traceback style
--tb=<style>

In conclusion
65
Code quality tools can help a lot with
Readability
Reducing bugs
Boosting developer productivity
Consistency
Refactoring
Teaching others good practice too
Hopefully we have had some helpful discussions!
It’s okay to disable a check
Try to understand why it’s there
Remember there are multiple concerns involved in decisions

Software Quality Assurance Tooling - Wintersession 2024

Recommended

Recommended

More Related Content

Similar to Software Quality Assurance Tooling - Wintersession 2024

Similar to Software Quality Assurance Tooling - Wintersession 2024 (20)

More from Henry Schreiner

More from Henry Schreiner (20)

Recently uploaded

Recently uploaded (20)

Software Quality Assurance Tooling - Wintersession 2024