PyCon Korea 2019 Tutorial

1. PyCon Korea 2019
pytest로 파이썬 코드 테스트하기
최영선
yeongseon.choe@gmail.com

2. PyCon Korea 2019
pytest

3. • Modify Code with Confidence
• Identify Bugs Early
• Improve System Design
https://alysivji.github.io/testing-101-introduction-to-testing.html#benefits-of-
testing
Why we should test code?

4. • python test framework.
• Advantages of pytest
• Easy to use
• Can run a specific test or subset of tests
• Skip tests
• Can run tests in parallel
What is pytest?

5. • Examples
• https://github.com/pallets/flask
What is pytest?

6. PyCon Korea 2019
1. Getting Started

7. • Requirements
• Python 3
• virtualenv
• Installation
Getting Started
$ virtualenv –p python3.7 venv
$ source virtualenv/bin/activate
$ (venv)

8. • An example of simple test
• Execution
# content of test_sample.py
def inc(x):
return x + 1
def test_answer():
assert int(3) == 5
$ pytest test_sample.py

9. • (venv) py.test test_sample.py
• (venv) py.test tests/test_sample.py
• (venv) py.test tests/
How to run test cases?
$ pytest -h
usage: pytest [options] [file_or_dir] [file_or_dir] [...]
positional arguments:
file_or_dir

10. Practice

11. # calculator.py
def add(x, y):
return x + y
def subtract(x, y):
return x - y
def multiply(x, y):
return x * y
def divide(x, y):
return x / y

12. # calculator.py
def add(x, y):
return x + y
def subtract(x, y):
return x - y
def multiply(x, y):
return x * y
def divide(x, y):
return x / y
# test_calculator.py
import pytest
from calculator import add, divide
def test_add():
assert add(1, 2) == 3
assert not add (2, 2) == 3
def test_divide():
with pytest.raises(ZeroDivisionError):
divide(1, 0)

13. PyCon Korea 2019
2. Project Structure

14. Structure of the Repository
• Run test
project/
sample/
__init__.py
sample.py
tests/
test_sample.py
$ cd project
$ python –m pytest tests/test_sample.py

15. Practice

16. $ cd pytest_tutorial
$ ls
calculator.py
test_calculator.py
$ cd pytest_tutorial
$ mkdir src
$ mkdir tests
$ mv caclculator.py src
$ mv test_calculator.py tests
$ pytest
# edit code
$ python –m pytest tests

17. PyCon Korea 2019
3. Fixture

18. Test Fixture
• A software test fixture sets up the system for the testing process by
providing it with all the necessary code to initialize it, thereby satisfying
whatever preconditions there may be.
• Frequently fixtures are created by handling setUp() and tearDown() events of
the unit testing framework.
Four phases of a test:
1. Set-up
2. Exercise, interacting with the system under test
3. Verify, determining whether the expected outcome has been
obtained
4. Tear down, to return to the original state

19. • Fixtures are run pytest before the actual test functions.
• @pytest.fixture() decorator
• Purpose
• Setup and Teardown for the tests (e.g, database)
• Test set for the tests
pytest fixtures

20. Practice

21. # calculator.py
class Calculator(object):
"""Calculator class"""
def __init__(self):
pass
@staticmethod
def add(a, b):
return a + b
@staticmethod
def subtract(a, b):
return a - b
@staticmethod
def multiply(a, b):
return a * b
@staticmethod
def divide(a, b):
return a / b

22. # calculator.py
class Calculator(object):
"""Calculator class"""
def __init__(self):
pass
@staticmethod
def add(a, b):
return a + b
@staticmethod
def subtract(a, b):
return a - b
@staticmethod
def multiply(a, b):
return a * b
@staticmethod
def divide(a, b):
return a / b
# test_calculator.py
from src.calculator import add
def test_add():
assert add(1, 2) == 3

23. # calculator.py
class Calculator(object):
"""Calculator class"""
def __init__(self):
pass
@staticmethod
def add(a, b):
return a + b
@staticmethod
def subtract(a, b):
return a - b
@staticmethod
def multiply(a, b):
return a * b
@staticmethod
def divide(a, b):
return a / b
# test_calculator.py
from src.calculator import Calculator
def test_add():
calculator = Calculator()
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
def test_subtract():
calculator = Calculator()
assert calculator.subtract(5, 1) == 4
assert calculator.subtract(3, 2) == 1
def test_multiply():
calculator = Calculator()
assert calculator.multiply(2, 2) == 4
assert calculator.multiply(5, 6) == 30
def test_divide():
calculator = Calculator()
assert calculator.divide(8, 2) == 4
assert calculator.divide(9, 3) == 3

24. # test_calculator.py
from src.calculator import Calculator
def test_add():
calculator = Calculator()
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
def test_subtract():
calculator = Calculator()
assert calculator.subtract(5, 1) == 4
assert calculator.subtract(3, 2) == 1
def test_multiply():
calculator = Calculator()
assert calculator.multiply(2, 2) == 4
assert calculator.multiply(5, 6) == 30
def test_divide():
calculator = Calculator()
assert calculator.divide(8, 2) == 4
assert calculator.divide(9, 3) == 3

25. # test_calculator.py
from src.calculator import Calculator
@pytest.fixture
def calculator():
calculator = Calculator()
return calculator
def test_add(calculator):
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
def test_subtract(calculator):
assert calculator.subtract(5, 1) == 4
assert calculator.subtract(3, 2) == 1
def test_multiply(calculator):
assert calculator.multiply(2, 2) == 4
assert calculator.multiply(5, 6) == 30

26. # confest.py
import pytest
from src.calculator import
Calculator
@pytest.fixture
def calculator():
calculator = Calculator()
return calculator
# test_calculator.py
from src.calculator import Calculator
@pytest.fixture
def calculator():
calculator = Calculator()
return calculator
def test_add(calculator):
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
def test_add_fail(calculator):
assert calculator.add(1, 2) != 6
assert calculator.add(2, 2) != 5
def test_multiply(calculator):
assert calculator.multiply(2, 2) == 4
assert calculator.multiply(5, 6) == 30

27. # test_calculator.py
from src.calculator import Calculator
def test_add(calculator):
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
def test_add_fail(calculator):
assert calculator.add(1, 2) != 6
assert calculator.add(2, 2) != 5

28. PyCon Korea 2019
4. Parameterize

29. pytest parameterize
• Define multiple sets of arguments and fixtures at the test function or class.
• @pytest.mark.parametrize: parametrizing test functions
• The builtin pytest.mark.parametrize decorator

30. Practice

31. # test_calculator.py
from src.calculator import Calculator
def test_add(calculator):
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
assert calculator.add(2, 2) == 4
pytest.mark.parametrize(argnames, argvalues)

32. # test_calculator.py
from src.calculator import Calculator
def test_add(calculator):
assert calculator.add(1, 2) == 3
assert calculator.add(2, 2) == 4
assert calculator.add(9, 2) == 11
@pytest.mark.parametrize(
"a, b, expected",
[(1, 2, 3),
(2, 2, 4),
(2, 7, 11)])
def test_add_fail(calculator, a, b, expected):
assert calculator.add(a, b) != expected

33. # test_calculator.py
from src.calculator import Calculator
@pytest.mark.parametrize(
"a, b, expected",
[(1, 2, 6),
(2, 2, 5),
(2, 7, 2)])
def test_add_fail(calculator, a, b, expected):
assert calculator.add(a, b) != expected
@pytest.mark.xfail(rason="wrong result")
@pytest.mark.parametrize(
"a, b, expected",
[(1, 2, 6),
(2, 2, 5),
(2, 7, 2)])
def test_add_fail(calculator, a, b, expected):
assert calculator.add(a, b) == expected

34. • Buildin markers: skip, skipif and fail
• skip: enable you to skip tests you don’t want to run
• @pytest.mark.skip(reason=‘something’)
• @pytest.mark.skipif(condition, reason=‘something’)
• xfail: we are telling pytest to run a test function, but we expect it to fail.
• @pytest.mark.fail

35. # test_calculator.py
from src.calculator import Calculator
@pytest.mark.xfail(rason="wrong result")
@pytest.mark.parametrize(
"a, b, expected",
[(1, 2, 6),
(2, 2, 5),
(2, 7, 2)])
def test_add_fail(calculator, a, b, expected):
assert calculator.add(a, b) == expected
@pytest.mark.parametrize(
"a, b, expected",
[pytest.param(1, 2, 6, marks=pytest.mark.xfail),
pytest.param(2, 2, 5, marks=pytest.mark.xfail),
pytest.param(2, 7, 2, marks=pytest.mark.xfail)])
def test_add_fail(calculator, a, b, expected):
assert calculator.add(a, b) == expected

36. # test_calculator.py
from src.calculator import Calculator
@pytest.mark.parametrize(
"a, b, expected",
[(1, 2, 3),
(2, 2, 4),
(2, 7, 9),
pytest.param(1, 2, 6, marks=pytest.mark.xfail),
pytest.param(2, 2, 5, marks=pytest.mark.xfail),
pytest.param(2, 7, 2, marks=pytest.mark.xfail)])
def test_add(calculator, a, b, expected):
assert calculator.add(a, b) == expected

37. # test_calculator.py
from src.calculator import Calculator
add_test_data = [
(1, 2, 3),
(2, 2, 4),
(2, 7, 9),
pytest.param(1, 2, 6, marks=pytest.mark.xfail),
pytest.param(2, 2, 5, marks=pytest.mark.xfail),
pytest.param(2, 7, 2, marks=pytest.mark.xfail)
]
@pytest.mark.parametrize(
"a, b, expected", add_test_data)
def test_add(calculator, a, b, expected):
assert calculator.add(a, b) == expected

38. • ids: optional parameter to parameterize()
• @pytest.mark.parameterize() decorator
• pytest.param(<value>, id=“something”)

39. # test_calculator.py
from src.calculator import Calculator
add_test_data = [
(1, 2, 3),
(2, 2, 4),
(2, 7, 9),
pytest.param(1, 2, 6, marks=pytest.mark.xfail),
pytest.param(2, 2, 5, marks=pytest.mark.xfail),
pytest.param(2, 7, 2, marks=pytest.mark.xfail)]
@pytest.mark.parametrize(
"a, b, expected", add_test_data, ids=[
"1 add 2 is 3",
"2 add 2 is 4",
"2 add 7 is 9",
"1 add 2 is not 6",
"2 add 2 is not 5",
"2 add 7 is not 2"] )
def test_add(calculator, a, b, expected):
assert calculator.add(a, b) == expected

40. # test_calculator.py
from src.calculator import Calculator
add_test_data = [
(1, 2, 3),
(2, 2, 4),
(2, 7, 9),
pytest.param(1, 2, 6, marks=pytest.mark.xfail),
pytest.param(2, 2, 5, marks=pytest.mark.xfail),
pytest.param(2, 7, 2, marks=pytest.mark.xfail)]
@pytest.mark.parametrize(
"a, b, expected", add_test_data, ids=[
"1 add 2 is 3",
"2 add 2 is 4",
"2 add 7 is 9",
"1 add 2 is not 6",
"2 add 2 is not 5",
"2 add 7 is not 2"] )
def test_add(calculator, a, b, expected):
assert calculator.add(a, b) == expected

41. # test_calculator.py
from src.calculator import Calculator
subtract_test_data = [
pytest.param(5, 1, 4, id="5 subtract 1 is 4"),
pytest.param(3, 2, 1, id="3 subtract 2 is 1"),
pytest.param(10, 2, 8, id="10 subtract 2 is 8"),
pytest.param(5, 1, 6, marks=pytest.mark.xfail, id="5 subtract 1 is 6"),
pytest.param(3, 2, 2, marks=pytest.mark.xfail, id="3 subtract 2 is 2"),
pytest.param(10, 2, 1, marks=pytest.mark.xfail, id="10 subtract 2 is 1")
]
@pytest.mark.parametrize(
"a, b, expected", subtract_test_data
)
def test_subtract(calculator, a, b, expected):
assert calculator.subtract(a, b) == expected

42. PyCon Korea 2019
5. Mock

43. • Thirty-party pytest plugin
• pytest-mock
• Swapping out part of the system to isolate bits of code
• Mock objects: test doubles, spies, fakes, or stubs..
• mock.patch
• mock.patch.object
pytest mock

44. Practice

45. • Installation
$ (venv) pip install pytest-mock

46. def test_add_with_mocker1(mocker, calculator):
"""Test functionality of add."""
mocker.patch.object(calculator, 'add', return_value=5)
assert calculator.add(1, 2) is 5
assert calculator.add(2, 2) is 5
def test_add_with_mocker2(mocker, calculator):
"""Test functionality of add."""
mocker.patch.object(calculator, 'add', side_effect=[1, 2])
assert calculator.add(1, 2) is 1
assert calculator.add(2, 2) is 2
def test_add_with_mocker3(mocker, calculator):
"""Test functionality of add."""
mocker.patch.object(calculator, 'add', side_effect=ZeroDivisionError())
with pytest.raises(ZeroDivisionError):
calculator.add(1, 2)

47. # calculator.py
import logging
logging.basicConfig(level=logging.INFO)
class Calculator():
"""Calculator class"""
def __init__(self):
self.logger = logging.getLogger(self.__class__.__name__)
def add(self, a, b):
self.logger.info(
"add {a} to {b} is {result}".format(
a=a, b=b, result=a + b
))
return a + b

48. @pytest.mark.parametrize(
"a, b, expected", add_test_data
)
# mocker: The mocker fixture is provided by the pytest-mock plugin
def test_add_spy_logger(mocker, calculator, a, b, expected):
spy_info = mocker.spy(calculator.logger, "info")
assert calculator.add(a, b) == expected
assert spy_info.called
assert spy_info.call_count == 1
calls = [mocker.call("add {a} to {b} is {expected}".format(
a=a, b=b, expected=expected
))]
assert spy_info.call_args_list == calls

49. References
• https://docs.pytest.org/en/latest/
• https://docs.python-guide.org/writing/structure/
• https://stackoverflow.com/questions/1896918/running-unittest-with-typical-
test-directory-structure
• https://www.slideshare.net/soasme/pytest
• https://github.com/pytest-dev/pytest-mock/