2017 Symfony day slides

1. Behat from zero to hero
“A practical guide to symfony integration and usage”
Matteo Moretti - Samuele Lilli@SymfonyDayIt - 20 Ottobre 2017 - Milano

2. Matteo Moretti | Samuele Lilli
WHO ARE WE?
Matteo Moretti Samuele Lilli

3. Matteo Moretti | Samuele Lilli
OUR
EXPERIENCE

4. Matteo Moretti | Samuele Lilli
OUR EXPERIENCE
ON nuvola.madisoft.it
https://symfony.com/showcase/191

5. Matteo Moretti | Samuele Lilli
BEHAVIOUR
DRIVEN
DEVELOPMENT

6. Matteo Moretti | Samuele Lilli
BEHAT

7. Matteo Moretti | Samuele Lilli
A LONG TIME
AGO ...

8. Matteo Moretti | Samuele Lilli

9. Matteo Moretti | Samuele Lilli

10. Matteo Moretti | Samuele Lilli
TESTS ARE GREEN!

12. Matteo Moretti | Samuele Lilli
BUT,
SUDDENLY...

13. Matteo Moretti | Samuele Lilli
AS SOON AS
YOU ADD MORE
TESTS...

14. Matteo Moretti | Samuele Lilli
HARD TO MAINTAIN

15. Matteo Moretti | Samuele Lilli
RANDOM
FAILURES

16. Matteo Moretti | Samuele Lilli
SLOW TESTS

18. Matteo Moretti | Samuele Lilli
“WHY THE HELL THIS
TEST IS BROKEN?
I DIDN’T MODIFY THAT
CODE”

19. Matteo Moretti | Samuele Lilli
“MMM, 10 MINUTES TO
ADD A FEATURE. 2
HOURS TO FIX TESTS.”

21. Matteo Moretti | Samuele Lilli

22. Matteo Moretti | Samuele Lilli

23. Matteo Moretti | Samuele Lilli
WHAT IS
BEHAT?

24. Matteo Moretti | Samuele Lilli
IT’S A BDD TOOL!

25. Matteo Moretti | Samuele Lilli
IT’S A
STORY
BDD TOOL!

26. Matteo Moretti | Samuele Lilli
WHAT IS A
STORY?

27. Matteo Moretti | Samuele Lilli
HI! I’M A STORY!

28. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

29. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

30. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

31. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

32. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

33. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

34. Matteo Moretti | Samuele Lilli
EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
ACCEPTANCE TEST
DOCUMENTATION THAT NEVER GOES OUT TO DATE
WRITTEN IN GHERKIN
EXECUTABLE SPECIFICATION
A STORY IS ...

35. Matteo Moretti | Samuele Lilli
GHERKIN

36. Matteo Moretti | Samuele Lilli
IT’S A LANGUAGE!

37. Matteo Moretti | Samuele Lilli
IT’S A BUSINESS
READABLE
DOMAIN SPECIFIC
LANGUAGE!

39. Matteo Moretti | Samuele Lilli
LIKE YAML
INDENTATION
RULES PARSING

40. Matteo Moretti | Samuele Lilli
STORIES ARE ORGANIZED IN
FEATURES FILES (.feature)

41. Matteo Moretti | Samuele Lilli
GIVEN - WHEN - THEN
(AND, BUT) ARE KEYWORDS
THAT MATCH PHP CODE
AND EXECUTE IT.
STEPS

42. Matteo Moretti | Samuele Lilli

44. Matteo Moretti | Samuele Lilli
REGEX

45. Matteo Moretti | Samuele Lilli

46. Matteo Moretti | Samuele Lilli

47. Matteo Moretti | Samuele Lilli

48. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

49. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

50. Matteo Moretti | Samuele Lilli
HOW A GOOD
STORY LOOKS
LIKE?

51. Matteo Moretti | Samuele Lilli
FOCUS ON OUTPUT NOT INPUT
X

52. Matteo Moretti | Samuele Lilli
√
FOCUS ON OUTPUT NOT INPUT

53. Matteo Moretti | Samuele Lilli
DON’T USE UI REFERENCES
X

54. Matteo Moretti | Samuele Lilli
DON’T USE UI REFERENCES
√

55. Matteo Moretti | Samuele Lilli
USE BACKGROUND

56. Matteo Moretti | Samuele Lilli
USE TRANSFORMERS

57. Matteo Moretti | Samuele Lilli
USE TRANSFORMERS

58. Matteo Moretti | Samuele Lilli
COLLAPSE STEPS AS MUCH AS
POSSIBLE
UNLESS THESE GIVES YOU
BUSINESS/STORY VALUE

59. Matteo Moretti | Samuele Lilli

60. Matteo Moretti | Samuele Lilli
HOW
MUCH
DETAILS?

61. Matteo Moretti | Samuele Lilli
“What many people don’t realize is that
different levels of details are appropriate
for different scenarios in the same system
- sometimes in the same feature -
depending on what it is they’re describing”
(The cucumber book - Matt Wynne, Aslak Hellesoy)

62. Matteo Moretti | Samuele Lilli
BEHAT → GHERKIN → FEATURES →
STORIES (SCENARIOS) → STEPS →
CONTEXTS

63. Matteo Moretti | Samuele Lilli
BEHAT → GHERKIN → FEATURES →
STORIES (SCENARIOS) → STEPS →
CONTEXTS

64. Matteo Moretti | Samuele Lilli
DIVIDE ET IMPERA

65. Matteo Moretti | Samuele Lilli
ONE FEATURE
PER .FEATURE FILE

66. Matteo Moretti | Samuele Lilli
DIVIDE FEATURES
AND CONTEXTS
BY DOMAIN SUBJECT

67. Matteo Moretti | Samuele Lilli
DIVIDE FEATURES
AND CONTEXTS
BY ACTIONS TO BE
PERFORMED ON
DOMAIN SUBJECTS

68. Matteo Moretti | Samuele Lilli

69. Matteo Moretti | Samuele Lilli

70. Matteo Moretti | Samuele Lilli
UI

71. Matteo Moretti | Samuele Lilli
HEADLESS BROWSER EMULATOR
VS
BROWSER CONTROLLER

72. Matteo Moretti | Samuele Lilli
HEADLESS BROWSER EMULATOR
● pure HTTP spec implementations
● very simple to run and configure
● no need of a real browser: speed
● usually no JS/AJAX support

73. Matteo Moretti | Samuele Lilli
BROWSER CONTROLLER
● aim to control a real browser
● complex to configure
● slow
● JS/AJAX support

75. Matteo Moretti | Samuele Lilli
YOU MAY NEED BOTH OF THEM

76. Matteo Moretti | Samuele Lilli
MINK
http://mink.behat.org/en/latest/

77. Matteo Moretti | Samuele Lilli
BEHAT PARSES SCENARIOS
+
MINK BROWSES THE WEB

79. Matteo Moretti | Samuele Lilli

80. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

81. Matteo Moretti | Samuele Lilli
HOW TO USE MINK
FROM CONTEXT
CLASSES?

82. Matteo Moretti | Samuele Lilli
PAGE OBJECT PATTERN
https://martinfowler.com/bliki/PageObject.html

85. Matteo Moretti | Samuele Lilli
https://github.com/sensiolabs/BehatPageObjectExtension
BEHAT PAGE OBJECT EXTENSION

86. Matteo Moretti | Samuele Lilli
ASSERTIONS

88. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

89. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

90. Matteo Moretti | Samuele Lilli
AJAX

91. Matteo Moretti | Samuele Lilli
DON’T PULL YOUR
TESTS WITH SLEEP()
CALLS

93. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

94. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

95. Matteo Moretti | Samuele Lilli
UI END-TO-END TESTS
● BRITTLE
● EXPENSIVE TO WRITE
● TIME CONSUMING TO RUN
● MORE AFFECTED BY NON-DETERMINISM

96. Matteo Moretti | Samuele Lilli

97. Matteo Moretti | Samuele Lilli
https://martinfowler.com/bliki/TestPyramid.html

98. Matteo Moretti | Samuele Lilli
DO YOU REALLY NEED A REAL JS BROWSER?
https://blog.eleven-labs.com/en/behat-structure-functional-tests/

99. Matteo Moretti | Samuele Lilli
STICK TO A BROWSER EMULATOR
USE @javascript ONLY WHEN NEEDED

100. Matteo Moretti | Samuele Lilli
FIXTURES DATA
● NO FIXTURES (INTEGRATION TESTS)
● LOAD FIXTURES AT RUNTIME
● IN MEMORY DBMS (SQLITE)

101. Matteo Moretti | Samuele Lilli
PARALLELISM
● SPEED UP PERFORMANCE
● COMPLEX SETUP AND DEBUG
DO YOU NEED IT?

102. Matteo Moretti | Samuele Lilli
FASTEST
https://github.com/liuggio/fastest
bin/behat --list-scenarios --config
tests/behat/behat.yml | bin/fastest -vv --process=4
"bin/behat --stop-on-failure --config
tests/behat/behat.yml {}"

103. Matteo Moretti | Samuele Lilli
ISSUES
HARD TO MAINTAIN
RANDOM FAILURES
SLOW TESTS

104. Matteo Moretti | Samuele Lilli
SAVE OUTPUT FOR
FAILED TESTS

106. Matteo Moretti | Samuele Lilli
DEBUG

107. Matteo Moretti | Samuele Lilli
CLI DEBUG
● BEHAT CODE
● APP CODE IF INTEGRATION TEST (NO UI)
● APP CODE IF UI TESTS WITH BROWSER
EMULATOR (BrowserKit)

108. Matteo Moretti | Samuele Lilli
WEB DEBUG
FOR APP CODE IF BROWSER CONTROLLER
(SELENIUM)

109. Matteo Moretti | Samuele Lilli
RECAP

110. Matteo Moretti | Samuele Lilli
TREAT YOUR TEST
CODE AS YOUR
APP (PROD) CODE

111. Matteo Moretti | Samuele Lilli
WRITE CLEAN,
CONCISE AND
WELL ORGANIZED
FEATURES

112. Matteo Moretti | Samuele Lilli
SPLIT YOUR
CONTEXT CLASSES
AND
COMPOSE / REUSE
THEM

113. Matteo Moretti | Samuele Lilli
REMEMBER TEST PYRAMID
FOCUS ON INTEGRATION LEVEL

114. Matteo Moretti | Samuele Lilli
USE @javascript WISELY
USE PAGE OBJECTS FOR UI

115. Matteo Moretti | Samuele Lilli
THANK YOU

116. Matteo Moretti | Samuele Lilli
WE ARE HIRING!
(wanna join? ask us or visit our website)
labs.madisoft.it

117. Matteo Moretti | Samuele Lilli
https://joind.in/talk/707a1

118. QUESTIONS?