2. Test Lint
• Test Lint
– Trustworthiness
– Maintainability
– Readability
• References
– Test Lint (http://docs.typemock.com/lint/ )
3. Trustworthy Tests
• Trustworthy Tests
– Always Asserts Something
– Avoid unpredictable factor
– Don’t depend on other tests
– Avoid Logic in Tests
4. Always Asserts Something
• Always Asserts Something
– Assertion means “Verification”
– Unit Tests need to “Check Something”
• Exceptions
– Check exception not thrown
• Name “Login_Call_NoExceptionThrown”
– Specified Mock Object
5. Avoid Unpredictable Factor
• Avoid Unpredictable Factor
– Unpredictable Factor
• Random Number
• Date Time
– Cause
• Unreliable/Inconsistent Result
• Hard to write Assertion statement
7. Don’t Depend on Other Tests
• Example
class LogAnalyzerDependTest extends PHPUnit_Framework_TestCase {
public function test_LogAnalyzerDepend_Construct_NoException() {
$this->analyzer = new LogAnalyzerDepend();
$this->analyzer->initialize();
}
public function test_IsValid_InValidContent_ReturnFalse() {
$this->test_LogAnalyzerDepend_Construct_NoException();
$this->assertFalse($this->analyzer->is_valid('abc'));
}
}
8. Don’t Depend on Other Tests
• Symptom
– A test calls another tests
– It requires other tests to create/delete objects
– A test depends on system state set by other tests
– Test order matters
9. Don’t Depend on Other Tests
• Why not?
– Cannot provide explicit testing result
– Implicit tests flow
– The testA failed because callee testB failed
• Solution
– Extract reused code to utility
– For create/delete object, use “setUp” and
“tearDown” instead
10. Avoid Logic in Tests
• Test more than one thing
– Number of Assertion > 1
– Logics better not in tests
• switch, if, or else statement
• foreach, for, or while loops
11. Avoid Logic in Tests- Example
• Test Code
public function test_ImplodeAndExplode_ValidContent_CorrectResult() {
$instance = new MoreThanOne();
$test_array = array('1', '2', '3');
$test_string = '1,2,3';
for ($i = 0; $i < 2; $i++) {
if ($i === 0) { // Test explode2
$result = $instance->explode2(',', $test_string);
$this->assertEquals($test_array, $result);
}
elseif ($i === 1) { // Test implode2
$result = $instance->implode2(',', $test_array);
$this->assertEquals($test_string, $result);
}
}
}
13. Trustworthiness – Do it
• Do it
– Testing only one thing
– Keep safe green zone
• No dependency to real database/network
• Keep result consistent
– Assuring code coverage
– Attitude
• Add a unit test for newly tickets/trackers
14. Maintainable Tests
• Maintainable Tests
– Test public function only
– Don’t Repeat Yourself
• Use Factory Function over Multiple Object Creation
• Use setUp
– Using setUp in a maintainable manner
– Avoid Over Specification in Tests
– Avoid multiple asserts
15. Test Public Function Only
• Test Public Function Only
– Design/Program by Contract
• Private/Protected method might be changed
– Extract private/protected function to new class
• Adopt Single Responsibility Principle (SRP)
16. Semantic Change
• Semantic Change is really a PAIN
– API change may break all tests
– Each test need to be changed
public function test_IsValid_InValidContent_ReturnFalse_New() {
$analyzer = new LogAnalyzer();
$analyzer->initialize(); // new requirement
$this->assertFalse($analyzer->is_valid('abc'));
}
17. Use Factory Function over Multiple
Object Creation
• Use a factory function
protected function create_LogAnalyzer() { // Factory Method
$analyzer = new LogAnalyzer();
$analyzer->initialize(); // New API handled here
return $analyzer;
}
public function test_IsValid_InValidContent_ReturnFalse() {
$analyzer = $this->create_LogAnalyzer(); // Use factory
$this->assertFalse($analyzer->is_valid('abc'));
}
18. Use setUp over Multiple Object
Creation
• Use setUp
protected function setUp() { // setUp method
$this->analyzer = new LogAnalyzer();
$this->analyzer->initialize(); // New API handled here
}
public function test_IsValid_InValidContent_ReturnFalse() {
$this->assertFalse($this->analyzer->is_valid('abc'));
}
public function test_IsValid_ValidContent_ReturnFalse() {
$this->assertTrue($this->analyzer->is_valid('valid'));
}
19. Maintainable setUp
• Maintainable setUp
– setUp() should be generic
• Cohesion
• Initialized object should be used in all tests
• Might not be appropriate to arrange mock/stub in
setUp()
– setUp() should be kept his readability
20. Avoid Over Specification in Test
• Over specified in test
– Verify internal state/behavior of object
– Using mocks when stubs are enough
– A test assumes specific order or exact string
matches when it isn’t required.
21. Verify internal state/behavior of object
• Solution
– Never verify internal state/behavior
– Maybe no need to test
public function
test_Initialize_WhenCalled_SetsDefaultDelimiterIsTabDelimiter(){
$analyzer = new LogAnalyzer();
$this->assertEquals(null,
$analyzer->GetInternalDefaultDelimiter()
);
$analyzer->Initialize();
$this->assertEquals('t',
$analyzer->GetInternalDefaultDelimiter()
);
}
22. Avoid Multiple Asserts
• Why not?
– Assertion failure will throw exception. Multiple
assertion cannot get all failure point at once
– Test multiple thing in one tests
• Solution
– Separate tests for different assertion
– Use data provider / parameter tests
23. Data Provider Sample
• Test Code
class DataTest extends PHPUnit_Framework_TestCase {
/**
* @dataProvider provider
*/
public function testAdd($a, $b, $c) {
$this->assertEquals($c, $a + $b);
}
public function provider() {
return array(
array(0, 0, 0),
array(0, 1, 1),
array(1, 0, 1),
array(1, 1, 3)
);
}
}
24. Readable Tests
• Test Naming
• Variable Naming
• Good Assert Message
• Separate Arrange and Assertion
• Mock and Stub Naming
25. Test Naming
• Function Name
– Test function name should be
test_<function>_<scenario>_<expect_behavior>
– Example
• test_escape_evenBackSlashesData_successEscape
26. Variable Name
• Avoid Hard Code in tests
public function test BadlyNamedTest() {
$log = new LogAnalyzer();
$result= log.GetLineCount("abc.txt");
$this->assertEquals(-100, result);
}
public function test WellNamedTest() {
$log = new LogAnalyzer();
$COULD_NOT_READ_FILE = -100;
$result= log.GetLineCount("abc.txt");
$this->assertEquals($COULD_NOT_READ_FILE, result);
}
27. Good Assertion Message
• Good Assertion Message
– Don’t repeat what the built-in test framework
outputs to the console.
– Don’t repeat what the test name explains.
– If you don’t have anything good to say, don’t say
anything.
– Write what should have happened or what failed
28. Separate Arrange and Assertion
• Separate Arrange and Assertion
public function test_BadAssertMessage() {
$this->assertEquals(COULD_NOT_READ_FILE,
log->GetLineCount("abc.txt")
);
}
public function test_GoodAssertMessage() {
$result = log->GetLineCount("abc.txt");
$this->assertEquals($COULD_NOT_READ_FILE, $result);
}
29. Mock and Stub Naming
• Include “mock” and “stub” in variable name
public function test_sendNotify_Mock_NoException() {
$notify_content = 'fake_content';
$mock_notifier = $this->getMock('NotifierInterface');
$mock_notifier->expects($this->once())
->method('notify')
->with($this->anything(),
$this->equalTo($notify_content));
$alert_system = new AlertSystem(
$mock_notifier,
$stub_provider
);
$alert_system->send_notify('Alert!!');
}
31. PHPUnit and Selenium
• Use PHPUnit to do
– Integration Test
– Acceptance Test
• References
– PHPUnit Selenium
• http://www.phpunit.de/manual/current/en/selenium.h
tml
• http://seleniumhq.org/documentation/
32. PHPUnit and Selenium
• Use PHPUnit and Selenium
class WebTest extends PHPUnit_Extensions_SeleniumTestCase {
protected function setUp() {
$this->setBrowser('*firefox');
$this->setBrowserUrl('http://www.example.com/');
}
public function testTitle() {
$this->open('http://www.example.com/');
$this->assertTitle('Example WWW Page');
}
}
