Introduction: How to use Unit Testing (TDD) and Behaviour Testing (BDD) for Wordpress, building a Continuous Integration workflow.

1. Effizientere
WordPress-Plugin-Entwicklung mit
Softwaretests
Martin Schütte

2. About DECK36
• Small team of 7 engineers
• Longstanding expertise in designing, implementing and operating
complex web systems
• Developing own data intelligence-focused tools and web services
• Offering our expert knowledge in Automation & Operation,
Architecture & Engineering, Analytics & Data Logistics

3. 1. Dev & Test Environments
2. Testing Variants
Static Code Analysis
Unit Testing
Integration Testing
Behaviour Testing
3. Integration & Automation

4. Main Questions
How can I know (my) software is correct?
How does my boss know software is correct?
How do I know software implements a given design?
How can we discuss what “correct” is, anyway?
We always need:
• implicit assumptions,
• explicit specifications.

5. Levels of Testing
..
abstract
.
specific
.
Unit Tests
.
Integration Tests
.
Acceptance
Tests

6. Example Plugin: Freifunkmeta

7. Use a Dev
Environment

8. Vagrant
Configuration tool for (VirtualBox) VM
setup and provisioning.
“Local cloud”
• Self service
• Instant provisioning
Useful for development
• reproducible environment
• independent PHP 5.x setups
• try things and not jeopardise
your dev environment

9. VagrantPress
$ git clone https://github.com/chad-thompson/vagrantpress.git
$ cd vagrantpress
$ vagrant up
will setup VM with:
• Ubuntu Precise (12.04), Apache 2.2, MySQL 5.5, PHP 5.3
• Wordpress 3.8
• phpMyAdmin
• PHPUnit
• phpcs, phploc, phpdepend, …

10. Testing Variants

11. Coding Style
$ phpcs --standard=WordPress freifunkmeta.php
FILE: [...]/plugins/freifunkmeta/freifunkmeta.php
---------------------------------------------------------------------
FOUND 360 ERROR(S) AND 406 WARNING(S) AFFECTING 338 LINE(S)
---------------------------------------------------------------------
21 | ERROR | Incorrect indentation; expected 1 space, found 4
22 | WARNING | Line is indented with space not tab
33 | ERROR | String "Unable to retrieve URL %s, error: %s" does
| | not require double quotes; use single quotes instead
322 | ERROR | Closing parenthesis of a multi-line function
| | definition must be on a line by itself
440 | ERROR | Expected next thing to be a escaping function,
| | not '"<option value='$city' $selected>$prettycity
| | </option>"'

12. Code Metrics
$ php pdepend_summary.php freifunkmeta.php
Package/Class Method LoC %Comment CCN NPath
------------- ------ --- -------- --- -----
FF_Meta output_ff_contact 49 6.1 10 384
FF_Meta shortcode_handler 41 2.4 9 48
FF_Community __construct 12 0.0 7 15625
FF_Meta register_stuff 18 0.0 5 16
FF_Meta aux_get_all_locations 23 8.7 5 6
FF_Community make_from_city 15 0.0 4 20
[...]

13. Example Plugin: Freifunkmeta
..WP Blog.
FF_Meta
.
WP Core
.
Other Plugins
.
FF_Community
.
FF_Dir
.
Output
Formatter
.
HTTP Get
Service

14. Unit Testing
..WP Blog.
FF_Meta
.
WP Core
.
Other Plugins
.
FF_Community
.
FF_Dir
.
Output
Formatter
.
HTTP Get
Service

15. Simple PHPUnit Test Case
class LowLevelTests extends PHPUnit_Framework_TestCase {
function setUp() {
$this->FFM = new FF_Meta();
}
function test_output_ff_state() {
$data = array("state" => array("nodes" => 429));
$ret = $this->FFM->output_ff_state($data);
$this->assertRegExp('/429/', $ret);
}
}

16. Unit Testing (contd.)
..WP Blog.
FF_Meta
.
WP Core
.
Other Plugins
.
FF_Community
.
FF_Dir
.
Output
Formatter
.
HTTP Get
Service

17. Integration Testing
..WP Blog.
FF_Meta
.
WP Core
.
Other Plugins
.
FF_Community
.
FF_Dir
.
Output
Formatter
.
HTTP Get
Service

18. Unit Testing with Mock Object
..WP Blog.
FF_Meta
.
WP Core
.
Other Plugins
.
FF_Community
.
FF_Dir
.
Output
Formatter
.
HTTP Get
Service

19. Example: Test with Dependency Injection
class MockDataService {
function get($url) {
return $some_fixed_data;
}
}
class WpIntegrationTests extends WP_UnitTestCase {
function setUp() {
parent::setUp();
// get plugin instance and replace ext. data service:
$this->plugin = $GLOBALS['wp-plugin-ffmeta'];
$this->plugin->reset_external_data_service(
new MockDataService() );
}
// ...

20. Example: Test with Dependency Injection
// ...
function test_post_ff_services() {
$post_attribs = array(
'post_title' => 'Test',
'post_content' => '[ff_services]' );
$post = $this->factory->post->create_and_get(
$post_attribs );
// w/o filter:
$this->assertEquals($post_content, $post->post_content);
// with filter:
$output = apply_filters( 'the_content',
$post->post_content );
$this->assertRegExp('/radio.ffhh/', $output);
}
}

21. PHPUnit Output

22. Behaviour Testing
..WP Blog.
FF_Meta
.
WP Core
.
Other Plugins
.
FF_Community
.
FF_Dir
.
Output
Formatter
.
HTTP Get
Service

23. WordPress Shortcode Plugin Test
..
Feature: Use Shortcodes
In order to use my Plugin
As a website author
I need to write posts with shortcodes
Background:
Given I am logged in as "admin" with "vagrant"
Scenario: Without the plugin
Given the plugin "freifunkmeta" is "inactive"
When I write a post with title "test" and content "[ff_contact]"
Then I should see "ff_contact"
Scenario: With the plugin
Given the plugin "freifunkmeta" is "active"
When I write a post with title "test" and content "[ff_contact]"
Then I should see "Twitter" in the ".ff_contact" element
And I should not see "ff_contact"

24. Behat Output

25. Implementation / Translation
A look behind the curtain:
• framework is clever but not magical
• some translation needed
• statements have to become executable code
Mechanism:
• plain sentence → method name
• quoted words → arguments
• matching with annotated regular expressions
• methods yield success, exception, or pending exception

26. Example: Behat Context (PHP)
/**
* from MinkContext
* Checks, that page contains specified text.
*
* @Then /^(?:|I )should see "(?P<text>(?:[^"]|")*)"$/
*/
public function assertPageContainsText($text)
{
$this->assertSession()->pageTextContains(
$this->fixStepArgument($text));
}

27. The Big Picture
..Features.
Step Definitions
.
WebDriver
.
Browser

28. The Big Picture
..Features.
Behat (PHP)
.
cucumber.js
.
Cucumber
(Ruby)
.
PhantomJS
.
Goutte
.
Selenium
.
Firefox
.
Chrome

29. Unit & Behaviour Testing
Unit Tests
• unit testing
• programmers
• programming language
• bottom-up
• assertXYZ
• tests derived from user stories
⇒ development tool
Behaviour Tests
• acceptance test scenarios
• non-developers
• language of business domain
• top-down / outside-in
• X should do Y
• execute user stories
⇒ design & communication tool

30. Automate!

31. Scripting
Basis for all automation.
Lots of useful builtins and packages:
• wp core download/install/config/…
• wp export/import
• wp plugin get/install/update/…
• wp scaffold _s/plugin/plugin-tests
• wp server

32. wp scaffold
Generate skeleton code for a new plugin & unit tests:
$ cd wordpress/wp-content/plugins
$ wp scaffold plugin-tests awesome
$ find awesome
awesome/
awesome/awesome.php
awesome/bin
awesome/bin/install-wp-tests.sh
awesome/tests
awesome/tests/bootstrap.php
awesome/tests/test-sample.php
awesome/.travis.yml
awesome/phpunit.xml

33. wp scaffold (contd.)
Create WP instance and run unit tests:
$ cd awesome
$ bash ./bin/install-wp-tests.sh wp_tests root vagrant latest
...
$ phpunit
PHPUnit 4.0.17 by Sebastian Bergmann.
[...]
Configuration read from [...]/plugins/awesome/phpunit.xml
.
Time: 5.52 seconds, Memory: 23.50Mb
OK (1 test, 1 assertion)

34. Version Control
• use version control!
• many possible workflows,
e. g. branches for dev and release
• use pre-commit hooks,
e. g. with php -l syntax check

35. Travis-CI
Continuous Integration service for GitHub
1. gets notified on push
2. builds project
3. runs phpunit
4. summarizes results,
alert on failure

36. Example .travis.yml
language: php
php:
- 5.4
- 5.5
- hhvm
env:
- WP_VERSION=3.8.3
- WP_VERSION=latest
before_script:
- bash bin/install-wp-tests.sh wordpress_test
root '' localhost $WP_VERSION
script: phpunit

37. Travis-CI Pass

38. Automated Testing
Target: no manual effort.
Continuous Integration:
• frequent code check-ins
• verified by automated
builds and tests
• quickly find bugs
and regressions
Continuous Deployment:
• (semi-)automated deployment
• plan for rollback

39. Costs and Benefits of Testing
• Testing (like Documentation) has a cost
• usually: productivity improvement > cost
• but find the right balance

40. Conclusion
I get paid for code that works, not for tests, so my philosophy is to test
as little as possible to reach a given level of confidence.
– Kent Beck
Links
• http://phpqatools.org/: PHP Quality Assurance Toolchain
• http://wpgear.org/: compendium of useful WP developer tools
• http://wptest.io/: test data for WP plugins and themes
• Ptah Dunbar: Automated Testing in WordPress, Really?!
• tuts+ articles by Tom McFarlin
• Conversation “Is TDD Dead?”

41. Thank You