Presentation is about how to automate native Android apps using Calabash-Driver and how to leverage an existing selenium grid architecture to allow scaling and parallel testing. Calabash-Driver Homepage: http://calabash-driver.github.com GDG DevFest Showcase Source Code: https://github.com/DominikDary/gdg-devfest-zrh

1. Calabash-Driver –
Test Automation for Android
Apps
Dominik Dary
Quality Engineering Europe
eBay International AG

2. FACTS ABOUT EBAY

3. EBAY INC. FACTS
•Founded in 1995
•Based in San Jose, California
•27770 Employees worldwide
Calabash-Driver – Test Automation for Android Apps 3

4. MOBILE FACTS
Mobile Facts Mobile Fun Facts
100,000,000 downloads Every 30 seconds, a woman’s handbag is purchased on
of eBay Inc’s apps eBay Mobile.
Mobile transaction volume Every 2 minutes, a tablet is bought through eBay Mobile.
will double in 2012 to 10 billion USD
1,7 million items Every week, more than 8,000 cars and 340,000 car parts
are listed with using mobile apps every week are sold through eBay’s mobile apps.
Calabash-Driver – Test Automation for Android Apps 4

5. WHO I AM
Dominik Dary
Staff Software Engineer in Test
E-mail: ddary@ebay.com
G+: Dominik Dary
Twitter: @DomeLG
Calabash-Driver – Test Automation for Android Apps 5

6. CHALLENGES IN
MOBILE TESTING

7. FIRST CHALLENGE: LOCALIZATION
Calabash-Driver – Test Automation for Android Apps 7

8. SECOND CHALLENGE: DEVICE DIVERSITY
Calabash-Driver – Test Automation for Android Apps 8

9. DEVICES VS. EMULATORS
Calabash-Driver – Test Automation for Android Apps 9

10. MANUAL MOBILE TESTING
•Feature testing is done manual
•Concentrate on top 5 devices
• Functionality, Layout, User Experience and
Localization
•In depth EU specific domain knowledge
Calabash-Driver – Test Automation for Android Apps 10

11. OUR REQUIREMENTS
FOR MOBILE TEST
AUTOMATION

12. THE IMPORTANCE OF PARALLEL TEST
EXECUTION
•Multiple Apps
•12 European Sites
•iOS Devices
–iPhone vs. iPad
–Version
•Android Devices
–Manufactures
–Screen Sizes
–OS Modifications
Picture Source: http://www.flickr.com/photos/ippei-janine/483345027/
Calabash-Driver – Test Automation for Android Apps 12

13. TECHNICAL HIGH LEVEL OVERVIEW OF
THE WEB TESTS
Calabash-Driver – Test Automation for Android Apps 13

14. MOBILE TEST AUTOMATION
REQUIREMENTS
•Use the Load Balancer Features of the Selenium
Grid2 to execute the End-to-End Tests in parallel
•Manage multiple applications / versions / languages
•The Mobile App should not need to be modified (e.g.
like adding a test server)
•Mobile App Runtime Inspection Support
Calabash-Driver – Test Automation for Android Apps 14

15. MOBILE TEST
AUTOMATION
IN PRACTICE

16. TESTING OF EBAY‘S MOBILE PRODUCTS
Android Mobile Web Android Native App iOS Native App
Calabash-Driver – Test Automation for Android Apps 16

17. TEST AUTOMATION OF THE
ANDROID APPS
•We use calabash-android on the device and have
created the Open Source project Calabash-Driver
•Calabash-Driver integrates the existing open source
project Calabash-Android into the Selenium Grid
•Calabash-Driver offers a convenient Webdriver like
Java API for writing tests
http://calabash-driver.github.com/
Calabash-Driver – Test Automation for Android Apps 17

18. TEST AUTOMATION OF ANDROID APPS
Selenium Grid
TestNG Hub
Test
http
http
Instrumentation
http Backend**
Calabash
Calabash Android
Jetty Server
Driver Connecto
r
eBay App
ADB
Connec- USB
tion
QE Machine Device
Calabash Server
Backend
** Component by the open source project Calabash-Android: http://calaba.sh
Calabash-Driver – Test Automation for Android Apps 18

19. CAPABILITIES ARE USED TO ENCAPSULATE
ANDROID PHONE DETAILS
Described by Capabilities:
"locale":"en_GB",
"deviceId":"emulator-5556",
"sdkVersion":"4.0.3",
"aut":"eBayMobile:1.8.1.5"
Instrumentation Instrumentation
Server Server
en_UK de_DE
Calabash-Driver Client Calabash-Driver Server
Selenium Grid Hub
Calabash-Driver – Test Automation for Android Apps 19

20. ANDROID APP TEST CASE
Tests
Flow Objects
Objects
Dialog
Calabash-Driver – Test Automation for Android Apps 20

21. GRID
INTEGRATION
DEMO
Calabash-Driver – Test Automation for Android Apps 21

22. HOW TO WRITE ROBUST END-TO-END TESTS
NATIVE IDS L10N SUPPORT
• Use the native ID like • Strings in the Android app can be
‘‘event_list_item_title” to interact with localized for different locales
an UI element (string.xml)
• driver.findViewById("event_list_item_titl • It is hard to maintain this strings in a
e").click(); multi locale test project
• Native ids are much more stable in • We use the localisation (l10n) key of
multi locale environments the resource file to interact with UI
elements
• driver.findL10nElement(By.l10nElement
("title_event_details")).waitFor();
Calabash-Driver – Test Automation for Android Apps 22

23. SPEED UP THE DEVELOPLMENT OF AUTOMATED
TESTS
INTERACTIVE RUBY SHELL DIALOG RUNTIME INSPECTION
• Interactive Command execution • Interactive Debugging of the current
• Results are immediately displayed screen
• The app can be used to navigate to the • Convenient way to retrieve the Element
dialog you want to automate. IDs
• Much Faster to use as Hierarchyviewer
Calabash-Driver – Test Automation for Android Apps 23

24. BUILD IN SUPPORT FOR WEBVIEWS
•Build in support for
entering text & click on
buttons
•CSS Selectors are used
•WebView can be
inspected for defining CSS
selectors
•API:
driver.webview(By.css(".button
#login")).click();
Calabash-Driver – Test Automation for Android Apps 24

25. SHOWCASE
https://github.com/DominikDary/gdg-devfest-zrh/
Calabash-Driver – Test Automation for Android Apps 25

26. ADVANTAGES OF USING
CALABASH-DRIVER
•Solution can be integrated into the Selenium Grid
which enables us to scale
•The concept of automated testing the Web, Win32,
mobile Web and native apps is the same
•Fast Rampup for the testers and quick return on
investment
•Calabash-Driver supports the interaction with
elements based on the Localization (L10n) resource
bundle key
Calabash-Driver – Test Automation for Android Apps 26

27. FORK ME AT GITHUB
http://calabash-driver.github.com
Calabash-Driver – Test Automation for Android Apps 27

28. CONCLUSION
•No single automation solution for mobile Web,
Android and iOS apps – and that’s ok!
•Invest into Scalability by Virtualization
•Buy the top devices
•Open source is your best friend!
Interested in Calabash-Driver? Contribute to our project!
Calabash-Driver – Test Automation for Android Apps 28

29. Thank You!
Questions or Comments?
WE ARE HIRING IN
BERLIN!
Sounds interesting for you?
http://ebaycareers.com Calabash-Driver – Test Automation for Android Apps
29