In the same way that we need to make assertions about how code functions, we need to make assertions about data, and unit testing is a promising framework. In this talk, we'll explore what is unique about unit testing data, and see how Two Sigma's open source library Marbles addresses these unique challenges in several real-world scenarios.
3. ONCE UPON A TIME...ONCE UPON A TIME...
CHILDREN WERE OLDER THAN THEIR PARENTS.CHILDREN WERE OLDER THAN THEIR PARENTS.
4. ONCE UPON A TIME...ONCE UPON A TIME...
CHILDREN WERE OLDER THAN THEIR PARENTS.CHILDREN WERE OLDER THAN THEIR PARENTS.
THEY WERETHEY WERE ALOTALOTOLDER THAN THEIR PARENTS.OLDER THAN THEIR PARENTS.
5. ONCE UPON A TIME...ONCE UPON A TIME...
CHILDREN WERE OLDER THAN THEIR PARENTS.CHILDREN WERE OLDER THAN THEIR PARENTS.
THEY WERETHEY WERE ALOTALOTOLDER THAN THEIR PARENTS.OLDER THAN THEIR PARENTS.
THEY WERE A LOT OLDER THANTHEY WERE A LOT OLDER THAN EVERYONEEVERYONE..
6. EVERYONE THAT WORKS WITH DATA HAS STORIES LIKEEVERYONE THAT WORKS WITH DATA HAS STORIES LIKE
THISTHIS
8. WHAT WERE MY ASSUMPTIONS?WHAT WERE MY ASSUMPTIONS?
9. WHAT WERE MY ASSUMPTIONS?WHAT WERE MY ASSUMPTIONS?
1. Children are born after their parents
10. WHAT WERE MY ASSUMPTIONS?WHAT WERE MY ASSUMPTIONS?
1. Children are born after their parents
2. People can't live forever
11. WHAT ELSE DO WE ASSUME ABOUT DATA?WHAT ELSE DO WE ASSUME ABOUT DATA?
12. WHAT ELSE DO WE ASSUME ABOUT DATA?WHAT ELSE DO WE ASSUME ABOUT DATA?
Values are correct
13. WHAT ELSE DO WE ASSUME ABOUT DATA?WHAT ELSE DO WE ASSUME ABOUT DATA?
Values are correct
We're not missing any data
14. WHAT ELSE DO WE ASSUME ABOUT DATA?WHAT ELSE DO WE ASSUME ABOUT DATA?
Values are correct
We're not missing any data
Records are unique
15. WHAT ELSE DO WE ASSUME ABOUT DATA?WHAT ELSE DO WE ASSUME ABOUT DATA?
Values are correct
We're not missing any data
Records are unique
Measurements are precise
16. WHAT ELSE DO WE ASSUME ABOUT DATA?WHAT ELSE DO WE ASSUME ABOUT DATA?
Values are correct
We're not missing any data
Records are unique
Measurements are precise
(this is a non-exhaustive list)
18. WE DON'T JUST HAVE DATA TO HAVE IT.WE DON'T JUST HAVE DATA TO HAVE IT.
19. WE DON'T JUST HAVE DATA TO HAVE IT.WE DON'T JUST HAVE DATA TO HAVE IT.
WE USE DATA TO MAKE DECISIONS.WE USE DATA TO MAKE DECISIONS.
20. WE SHOULD BE EXPLICIT ABOUT OUR ASSUMPTIONS.WE SHOULD BE EXPLICIT ABOUT OUR ASSUMPTIONS.
21. WHAT ARE THE IMPORTANT PROBLEMS HERE?WHAT ARE THE IMPORTANT PROBLEMS HERE?
22. WHAT ARE THE IMPORTANT PROBLEMS HERE?WHAT ARE THE IMPORTANT PROBLEMS HERE?
Data are always changing
23. WHAT ARE THE IMPORTANT PROBLEMS HERE?WHAT ARE THE IMPORTANT PROBLEMS HERE?
Data are always changing
Some changes are loud while others are silent
24. WHAT ARE THE IMPORTANT PROBLEMS HERE?WHAT ARE THE IMPORTANT PROBLEMS HERE?
Data are always changing
Some changes are loud while others are silent
Manually checking data is inconsistent and error-prone
25. WHAT ARE THE IMPORTANT PROBLEMS HERE?WHAT ARE THE IMPORTANT PROBLEMS HERE?
Data are always changing
Some changes are loud while others are silent
Manually checking data is inconsistent and error-prone
We're working with alotof data
26. WHAT ARE THE IMPORTANT PROBLEMS HERE?WHAT ARE THE IMPORTANT PROBLEMS HERE?
Data are always changing
Some changes are loud while others are silent
Manually checking data is inconsistent and error-prone
We're working with alotof data
We're working with a lot of differentkindsof data
29. WHAT DO WE WANT TO DO?WHAT DO WE WANT TO DO?
Encode our assumptions in testable form
30. WHAT DO WE WANT TO DO?WHAT DO WE WANT TO DO?
Encode our assumptions in testable form
Test those assumptions on incoming data
31. WHAT DO WE WANT TO DO?WHAT DO WE WANT TO DO?
Encode our assumptions in testable form
Test those assumptions on incoming data
Report when our assumptions don't hold
32. WHAT DO WE WANT TO DO?WHAT DO WE WANT TO DO?
Encode our assumptions in testable form
Test those assumptions on incoming data
Report when our assumptions don't hold
Report allof the assumptions that don't hold
35. HOW DOES UNITTEST SOLVE OUR PROBLEM?HOW DOES UNITTEST SOLVE OUR PROBLEM?
36. HOW DOES UNITTEST SOLVE OUR PROBLEM?HOW DOES UNITTEST SOLVE OUR PROBLEM?
Encode our assumptions in testable form
37. HOW DOES UNITTEST SOLVE OUR PROBLEM?HOW DOES UNITTEST SOLVE OUR PROBLEM?
Test those assumptions on incoming data
Encode our assumptions in testable form
38. HOW DOES UNITTEST SOLVE OUR PROBLEM?HOW DOES UNITTEST SOLVE OUR PROBLEM?
Report when our assumptions don't hold
Encode our assumptions in testable form
Test those assumptions on incoming data
39. HOW DOES UNITTEST SOLVE OUR PROBLEM?HOW DOES UNITTEST SOLVE OUR PROBLEM?
Report allof the assumptions that don't hold
Encode our assumptions in testable form
Test those assumptions on incoming data
Report when our assumptions don't hold
40. tripduration 0 days 00:18:09
starttime 2018-08-01 00:00:09.341000-04:00
stoptime 2018-08-01 00:18:18.889000-04:00
start station id 31
start station name Seaport Hotel - Congress St at Seaport Ln
start station latitude 42.3488
start station longitude -71.0417
end station id 190
end station name Nashua Street at Red Auerbach Way
end station latitude 42.3657
end station longitude -71.0643
bikeid 1026
usertype Subscriber
birth year 1969
gender 0
Hubway Bike Share Dataset
41. How long was the trip?
tripduration 0 days 00:18:09
starttime 2018-08-01 00:00:09.341000-04:00
stoptime 2018-08-01 00:18:18.889000-04:00
start station id 31
start station name Seaport Hotel - Congress St at Seaport Ln
start station latitude 42.3488
start station longitude -71.0417
end station id 190
end station name Nashua Street at Red Auerbach Way
end station latitude 42.3657
end station longitude -71.0643
bikeid 1026
usertype Subscriber
birth year 1969
gender 0
Hubway Bike Share Dataset
42. How far was the trip?
tripduration 0 days 00:18:09
starttime 2018-08-01 00:00:09.341000-04:00
stoptime 2018-08-01 00:18:18.889000-04:00
start station id 31
start station name Seaport Hotel - Congress St at Seaport Ln
start station latitude 42.3488
start station longitude -71.0417
end station id 190
end station name Nashua Street at Red Auerbach Way
end station latitude 42.3657
end station longitude -71.0643
bikeid 1026
usertype Subscriber
birth year 1969
gender 0
Hubway Bike Share Dataset
43. Internal metadata
tripduration 0 days 00:18:09
starttime 2018-08-01 00:00:09.341000-04:00
stoptime 2018-08-01 00:18:18.889000-04:00
start station id 31
start station name Seaport Hotel - Congress St at Seaport Ln
start station latitude 42.3488
start station longitude -71.0417
end station id 190
end station name Nashua Street at Red Auerbach Way
end station latitude 42.3657
end station longitude -71.0643
bikeid 1026
usertype Subscriber
birth year 1969
gender 0
Hubway Bike Share Dataset
44. Who took the trip?
tripduration 0 days 00:18:09
starttime 2018-08-01 00:00:09.341000-04:00
stoptime 2018-08-01 00:18:18.889000-04:00
start station id 31
start station name Seaport Hotel - Congress St at Seaport Ln
start station latitude 42.3488
start station longitude -71.0417
end station id 190
end station name Nashua Street at Red Auerbach Way
end station latitude 42.3657
end station longitude -71.0643
bikeid 1026
usertype Subscriber
birth year 1969
gender 0
Hubway Bike Share Dataset
45. ???
tripduration 0 days 00:18:09
starttime 2018-08-01 00:00:09.341000-04:00
stoptime 2018-08-01 00:18:18.889000-04:00
start station id 31
start station name Seaport Hotel - Congress St at Seaport Ln
start station latitude 42.3488
start station longitude -71.0417
end station id 190
end station name Nashua Street at Red Auerbach Way
end station latitude 42.3657
end station longitude -71.0643
bikeid 1026
usertype Subscriber
birth year 1969
gender 0
Hubway Bike Share Dataset
46. class TripDistanceTestCase(unittest.TestCase):
def setUp(self):
self.data = ...
def tearDown(self):
delattr(self, 'data')
def test_for_long_trips(self):
thresholds = [
('marathon', 42195),
('10km', 10000)
]
for severity, threshold in thresholds:
with self.subTest(severity=severity):
long_trips = self.data[
self.data['distance_meters'] > threshold]
self.assertTrue(long_trips.empty)
47. Load the data
class TripDistanceTestCase(unittest.TestCase):
def setUp(self):
self.data = ...
def tearDown(self):
delattr(self, 'data')
def test_for_long_trips(self):
thresholds = [
('marathon', 42195),
('10km', 10000)
]
for severity, threshold in thresholds:
with self.subTest(severity=severity):
long_trips = self.data[
self.data['distance_meters'] > threshold]
self.assertTrue(long_trips.empty)
48. Pick some thresholds
class TripDistanceTestCase(unittest.TestCase):
def setUp(self):
self.data = ...
def tearDown(self):
delattr(self, 'data')
def test_for_long_trips(self):
thresholds = [
('marathon', 42195),
('10km', 10000)
]
for severity, threshold in thresholds:
with self.subTest(severity=severity):
long_trips = self.data[
self.data['distance_meters'] > threshold]
self.assertTrue(long_trips.empty)
49. For each threshold
class TripDistanceTestCase(unittest.TestCase):
def setUp(self):
self.data = ...
def tearDown(self):
delattr(self, 'data')
def test_for_long_trips(self):
thresholds = [
('marathon', 42195),
('10km', 10000)
]
for severity, threshold in thresholds:
with self.subTest(severity=severity):
long_trips = self.data[
self.data['distance_meters'] > threshold]
self.assertTrue(long_trips.empty)
50. Find trips longer than the threshold
class TripDistanceTestCase(unittest.TestCase):
def setUp(self):
self.data = ...
def tearDown(self):
delattr(self, 'data')
def test_for_long_trips(self):
thresholds = [
('marathon', 42195),
('10km', 10000)
]
for severity, threshold in thresholds:
with self.subTest(severity=severity):
long_trips = self.data[
self.data['distance_meters'] > threshold]
self.assertTrue(long_trips.empty)
51. Assert none exist
class TripDistanceTestCase(unittest.TestCase):
def setUp(self):
self.data = ...
def tearDown(self):
delattr(self, 'data')
def test_for_long_trips(self):
thresholds = [
('marathon', 42195),
('10km', 10000)
]
for severity, threshold in thresholds:
with self.subTest(severity=severity):
long_trips = self.data[
self.data['distance_meters'] > threshold]
self.assertTrue(long_trips.empty)
52. $ python -m unittest test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/leif/test_bikeshare.py", line 107, in test_for_long_trips
self.assertTrue(long_trips.empty)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
54. WHAT DOES THIS GIVE US?WHAT DOES THIS GIVE US?
New data are automatically tested for long trips
55. WHAT DOES THIS GIVE US?WHAT DOES THIS GIVE US?
New data are automatically tested for long trips
Don't have to remember howto test for long trips
56. WHAT DOES THIS GIVE US?WHAT DOES THIS GIVE US?
New data are automatically tested for long trips
Don't have to remember howto test for long trips
Can easily run this test over historical data
59. WE'RE IN A PRETTY GOOD SPOT!WE'RE IN A PRETTY GOOD SPOT!
60. WE'RE IN A PRETTY GOOD SPOT!WE'RE IN A PRETTY GOOD SPOT!
1. We've thought through our assumptions about the data
61. WE'RE IN A PRETTY GOOD SPOT!WE'RE IN A PRETTY GOOD SPOT!
1. We've thought through our assumptions about the data
2. We've made them explicit by writing them down
62. WE'RE IN A PRETTY GOOD SPOT!WE'RE IN A PRETTY GOOD SPOT!
1. We've thought through our assumptions about the data
2. We've made them explicit by writing them down
3. We've made them executable
63. WE'RE IN A PRETTY GOOD SPOT!WE'RE IN A PRETTY GOOD SPOT!
1. We've thought through our assumptions about the data
2. We've made them explicit by writing them down
3. We've made them executable
4. We've automated them
64. WE'RE IN A PRETTY GOOD SPOT!WE'RE IN A PRETTY GOOD SPOT!
1. We've thought through our assumptions about the data
2. We've made them explicit by writing them down
3. We've made them executable
4. We've automated them
⭐⭐
66. $ python -m unittest test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/leif/test_bikeshare.py", line 107, in test_for_long_trips
self.assertTrue(long_trips.empty)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
67. Her: "Is there a way to see local variables in my unittest output?"
$ python -m unittest test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/leif/test_bikeshare.py", line 107, in test_for_long_trips
self.assertTrue(long_trips.empty)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
68. Her: "Is there a way to see local variables in my unittest output?"
Him: "I think pytest does that..."
$ python -m unittest test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/leif/test_bikeshare.py", line 107, in test_for_long_trips
self.assertTrue(long_trips.empty)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
69. PUT YOURSELF IN THE TEST CONSUMER'S SHOESPUT YOURSELF IN THE TEST CONSUMER'S SHOES
70. PUT YOURSELF IN THE TEST CONSUMER'S SHOESPUT YOURSELF IN THE TEST CONSUMER'S SHOES
What is this test doing? Why is it here?
71. PUT YOURSELF IN THE TEST CONSUMER'S SHOESPUT YOURSELF IN THE TEST CONSUMER'S SHOES
What is this test doing? Why is it here?
What am I supposed to do about this failure?
72. PUT YOURSELF IN THE TEST CONSUMER'S SHOESPUT YOURSELF IN THE TEST CONSUMER'S SHOES
What is this test doing? Why is it here?
What am I supposed to do about this failure?
How bad is it?
73. PUT YOURSELF IN THE TEST CONSUMER'S SHOESPUT YOURSELF IN THE TEST CONSUMER'S SHOES
What is this test doing? Why is it here?
What am I supposed to do about this failure?
How bad is it?
Have we seen this failure before? When?
74. PUT YOURSELF IN THE TEST CONSUMER'S SHOESPUT YOURSELF IN THE TEST CONSUMER'S SHOES
What is this test doing? Why is it here?
What am I supposed to do about this failure?
How bad is it?
Have we seen this failure before? When?
CONTEXT IS EXPENSIVE TO RECOVERCONTEXT IS EXPENSIVE TO RECOVER
75. THIS ISN'T UNIQUE TO DATA,THIS ISN'T UNIQUE TO DATA,
BUT IT'S ESPECIALLY HARD WITH DATABUT IT'S ESPECIALLY HARD WITH DATA
76. THIS ISN'T UNIQUE TO DATA,THIS ISN'T UNIQUE TO DATA,
BUT IT'S ESPECIALLY HARD WITH DATABUT IT'S ESPECIALLY HARD WITH DATA
1. Assumptions aren't black-and-white
77. THIS ISN'T UNIQUE TO DATA,THIS ISN'T UNIQUE TO DATA,
BUT IT'S ESPECIALLY HARD WITH DATABUT IT'S ESPECIALLY HARD WITH DATA
1. Assumptions aren't black-and-white
2. Failures are usually introduced by someone else
78. THIS ISN'T UNIQUE TO DATA,THIS ISN'T UNIQUE TO DATA,
BUT IT'S ESPECIALLY HARD WITH DATABUT IT'S ESPECIALLY HARD WITH DATA
1. Assumptions aren't black-and-white
2. Failures are usually introduced by someone else
3. Different tests require different follow-up
79.
80. ANATOMY OF A MARBLES FAILURE MESSAGEANATOMY OF A MARBLES FAILURE MESSAGE
81. $ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
82. What is this test doing?
$ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
83. What is this test doing?
$ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
84. Why is it here?
$ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
85. What am I supposed to do about this failure?
$ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
86. How bad is it?
$ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
87. Can we add more context?
$ python -m marbles test_bikeshare.py
======================================================================
FAIL: test_for_long_trips (test_bikeshare.TripDistanceTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: False is not true
Source (/home/leif/test_bikeshare.py):
151 self.data['distance_meters'] > threshold]
> 152 self.assertTrue(long_trips.empty, note=note)
153
Locals:
severity = 'marathon'
threshold = 42195
long_trips =
start station latitude stop station latitude ...
27955 42.366277 0.0 ...
Note:
There appear to be some trips in the data that are longer than a
marathon! If these are legitimate trips, consider contacting the
local news station about a human-interest story. If these do not
appear to be legitimate trips, contact the bike share mechanics
to have affected bikes identified and repaired.
92. SEMANTIC ASSERTIONSSEMANTIC ASSERTIONS
self.assertTrue((lower < x) and (x < upper))
self.assertGreater(x, lower)
self.assertGreater(upper, x)
self.assertTrue(all(a < b for a, b in zip([lower, x], [x, upper])))
self.assertBetween(x, lower, upper)
100. DOES MARBLES RECOVER THE CONTEXT WE WANTED?DOES MARBLES RECOVER THE CONTEXT WE WANTED?
101. DOES MARBLES RECOVER THE CONTEXT WE WANTED?DOES MARBLES RECOVER THE CONTEXT WE WANTED?
What is this test doing? Why is it here?
102. DOES MARBLES RECOVER THE CONTEXT WE WANTED?DOES MARBLES RECOVER THE CONTEXT WE WANTED?
What am I supposed to do about this failure?
What is this test doing? Why is it here?
103. DOES MARBLES RECOVER THE CONTEXT WE WANTED?DOES MARBLES RECOVER THE CONTEXT WE WANTED?
How bad is it?
What is this test doing? Why is it here?
What am I supposed to do about this failure?
104. DOES MARBLES RECOVER THE CONTEXT WE WANTED?DOES MARBLES RECOVER THE CONTEXT WE WANTED?
Have we seen this failure before? When?
What is this test doing? Why is it here?
What am I supposed to do about this failure?
How bad is it?
118. AGGREGATE DATASET HEALTH METRICSAGGREGATE DATASET HEALTH METRICS
df = df.pivot_table(
index=['month'], columns=['severity'],
values='anomalies', aggfunc=sum)
df.describe()
119. AGGREGATE DATASET HEALTH METRICSAGGREGATE DATASET HEALTH METRICS
df = df.pivot_table(
index=['month'], columns=['severity'],
values='anomalies', aggfunc=sum)
df.describe()
120. CONTEXT IS GOOD FOR SOFTWARE TESTS, TOOCONTEXT IS GOOD FOR SOFTWARE TESTS, TOO
121. CONTEXT IS GOOD FOR SOFTWARE TESTS, TOOCONTEXT IS GOOD FOR SOFTWARE TESTS, TOO
$ python -m unittest
F
======================================================================
FAIL: test_return_code (docs.examples.getting_started.ResponseTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/leif/git/marbles/docs/examples/getting_started.py", line 43, in test_return_code
201
AssertionError: 409 != 201
----------------------------------------------------------------------
Ran 1 test in 0.000s
122. CONTEXT IS GOOD FOR SOFTWARE TESTS, TOOCONTEXT IS GOOD FOR SOFTWARE TESTS, TOO
$ python -m marbles
F
======================================================================
FAIL: test_return_code (docs.examples.getting_started.ResponseTestCase)
----------------------------------------------------------------------
marbles.core.marbles.ContextualAssertionError: 409 != 201
Source (/home/leif/git/marbles/docs/examples/getting_started.py):
40 res = requests.put(endpoint, data=data)
> 41 self.assertEqual(
42 res.status_code,
43 201
44 )
Locals:
endpoint = 'http://example.com/api/v1/resource'
data = {'id': 1, 'name': 'Little Bobby Tables'}
res = <docs.examples.getting_started.Response object at 0x7fae97e78978>
----------------------------------------------------------------------
Ran 1 test in 0.001s
123. TWO STEPS TO MARBLESTWO STEPS TO MARBLES
$ pip install marbles
$ python -m marbles test_module.py