Adding Tests#

Ensure you have installed the development dependencies by following the instructions in the installation guide. To run all tests, use the following command:

invoke test

PyTest#

Tests can be added to the tests directory. PyTest will automatically discover and run these tests. They should be named test_*.py, and the test functions should be named test_*. See the PyTest documentation for more information.

Use PyTest style tests for parameterised tests, tests that require fixtures, and tests that require setup.

These tests can be run with:

pytest

Or to run a specific test

pytest tests/test_*.py

Or to run with the same configuration as continuous integration:

invoke test.pytest

Doctest#

Doctest allows you to write tests directly in the docstrings of your code, making it easier to keep documentation up-to-date. The tests are written as examples in a Python interactive shell.

Use doctest style tests to document code with simple tested examples.

Here’s an example of a function with a doctest:

def hello_world():
    """
    >>> hello_world()
    Hello, World!
    """
    print("Hello, World!")

You can run this test with:

pytest --doctest-modules path/to/your/module.py

Alternatively, you can run all unit tests with the same configuration as continuous integration:

invoke test.doctest

Notebooks#

We use nbmake to test that all notebooks in the notebooks directory run without error. This ensures that the notebooks are always up-to-date and working correctly.

You can run a notebook as a test with:

pytest --nbmake notebooks/my_notebook.ipynb

# Often the examples take too long to run regularly as tests. To speed up testing some
# notebooks use the `NB_FAST` environment variable to run the notebook faster by using
# smaller datasets or fewer iterations. To run them in this mode use:
NB_FAST=true pytest --nbmake notebooks/my_notebook.ipynb

For more about NB_FAST read the notebooks documentation.

Code Coverage#

Code coverage measures how many statements of code is executed while running tests. It identifies unused and untested code. We encourage contributors to use it to write more robust programs, but don’t have a target percantage.

To generate code coverage reports add --cov=capymoa and --cov-report=html to the pytest command:

pytest --cov=capymoa --cov-report=html

Alternatively, CapyMOA’s invoke testing tasks can generate coverage reports with:

invoke test --coverage