A tale about mobile test automation

1. @eddybruin
@smaljaars
#yvonne

2. 22 mei 2013 2

3. 22 mei 2013 3

6. 22 mei 2013 6

7. 22 mei 2013 7

8. iOS Mobile App @ ING Netherlands
• Developed in-house
• Working on 11th major release
since v1.0 November 2011
• Customized User Interface
• Native application
• Customer feedback is pivotal in
release planning
8

9. 22 mei 2013 9

10. Challenges in mobile testing
10

11. 11

12. 22 mei 2013 12

13. 13

14. 13
Top 6 Android users by February 2013
User analytics based testing

15. 14
Device Marketshare
Source: http://developer.android.com/about/dashboards/index.html
Usage of Android versions – March 2013
User analytics based testing

16. 14
Device Marketshare
Source: http://chitika.com/ios-version-distribution /
Usage of iOS versions – March 2013
User analytics based testing

17. 15
1
Full force manual testing

19. 22 mei 2013 17
2
There is
no silver
bullet (yet)
• 1 Tool
• Test all device, OS combinations
• 100% tested
• Low cost
• Quick results
• Easy maintainable
• Easy to learn
What does management want?

20. Mobile Test Automation Eco-system
22 mei 2013 18
Technique
Framework
Infrastructure

21. Mobile Test Automation Eco-system
22 mei 2013 52
• Technique to interact with the (web) app
• OCR vs. Code elements
Technique

22. Mobile Test Automation Eco-system
22 mei 2013 53
• Maintainability
• ATDD
Framework

23. Mobile Test Automation Eco-system
22 mei 2013 54
• Deal with fragmentation
• Test management
• Cloud vs. private
• Automation vs. manual
• Extra test coverage
Infrastructure

24. Mobile Test Automation Eco-system
22 mei 2013 18
Technique
Framework
Infrastructure
3
Mobile test automation eco-system =
Technique, Framework and
Infrastructure tools

25. 22 mei 2013 19
Technique
InfrastructureFramework
Device coverage

26. 22 mei 2013 20
Technique
InfrastructureFramework
Platform coverage

27. 22 mei 2013 21

28. 22 mei 2013 22
Technique
Framework
Infra.

29. Robotium & UIautomator

30. Experitest / Silk mobile

31. Experitest / Silk mobile

32. 22 mei 2013 26

33. 22 mei 2013 26
4
Onion structure of UIA code
enables maintainability,
accessability and readability

35. Test case: should make payment
Test steps:
Actions: tap, fill
Elements: amount input field
Test data
1
2
3
4
Amount
Creditor name
Account number
Description
1
2
3
4
fill fields &
send payment

36. • Enforced consistency coding
• Test data central location
• Pixel perfect testing
• Exporting logging
22 mei 2013 29
Cooking the onion

37. 22 mei 2013 30
Technique
Framework
Infra.
KIF
Frank

39. 22 mei 2013 32

40. 22 mei 2013 33
5

41. 22 mei 2013 34

42. 22 mei 2013 35
OPSPM
DEV QA

43. 22 mei 2013 36
6 Yvonne guarantees test
coverage over multiple iOS
devices and OS versions

44. 22 mei 2013 37

45. 22 mei 2013 38
Technique
Framework
Infra.
KIF
Frank

46. 39
Use the cloud

47. 42
Use the crowd

48. 22 mei 2013 43
Technique
Framework
Infra.
KIF
Frank

49. 22 mei 2013 44
Technique
Framework
Infra.

50. 22 mei 2013 45
Picture creativity
7

51. Yvonne half a year later….
22 mei 2013 46

52. Tweet aways Mobile testing
22 mei 2013 47
1
2
3
4
5
6
7
Exhaustive manual testing is impossible
There is no silver bullet (yet)
Mobile test automation eco-system =
Technique, Framework and Infrastructure tools
Structure your UIA code
Continuous integration powerful enabler
Test automation = team effort
Yvonne guarantees test coverage over
multiple iOS devices and OS versions

53. 22 mei 2013 48
@eddybruin
@smaljaars
#yvonne
http://goo.gl/95Tvp
Yvonne’s tale continues…