This presentation was presented in Android Only! 2011 conference on June 14th. With more than 300 different Android devices out on 6 different platform versions, application developers are facing a real nightmare when trying validate that their applications really work on their customers' devices. While fragmentation is a new thing in Android platform, it is not new in software industry and there are several ways to deal with device fragmentation from testing point of view. This presentation discusses most common approaches to tackle fragmentation from application developer's point of view and explains why testing for device compatibility is a must for any serious Android application developer.

1. Strategies for coping with device fragmentation

2. About Bitbar A technology company specialized in creating tools and environments for Android testing Experience ranging from hosting the build and test process of the whole Android stack to creating custom application testing device clusters Products: Testdroid Recorder and Testdroid Server

3. What problem are we solving?

4. Never able to log in! Uninstalling… Galaxy S Hung phone, drains power (Inspire) Real junk, although installed would not run Doesn’t work half of the time – HTC Evo Why is it important? Does not work on Motorola Zoom I cannot get this program to uninstall There seems to be more bugs than before update Xperia X10 - Widget doesn’t update and crashes all the time Could not connect using my Incredible

5. Approaches for managing this problem 1. Do nothing 2. Test manually every time you publish a new version 3. Outsource your manual testing to low cost countries 4. Automate your tests and run those on real hardware every time the code is changed

6. Frameworks for automated Android testing Monkey – a random UI exerciser tool Monkeyrunner – Jython based test scripting tool Robotium – It’s like Selenium but for Android Roboelectric – Unit testing for Android applications Cucumber – scenario based testing implemented in Ruby

7. Monkey A UI/Application exerciser monkey for creating stream of random user events such as clicks, touches, gestures and some system level events Useful way to stress test your application to find non-obvious usage patterns and corner cases $ adb shell monkey -pyour.package.name -v 500

8. Monkeyrunner An API toolkit for writing programs that control an Android device or emulator from outside of Android code # Runs the component device.startActivity(component=runComponent) # Presses the Menu button device.press('KEYCODE_MENU','DOWN_AND_UP') # Takes a screenshot result = device.takeSnapshot() # Writes the screenshot to a file result.writeToFile('myproject/shot1.png','png')

9. Robotium Powerful and robust automatic black-box test cases for Android applications Function, system and acceptance test scenarios spanning multiple Android activities @Smoke public void AddNewHudson() throws Exception { solo.clickOnText("Configuration"); solo.enterText(0, "hudson:8080"); solo.enterText(1, "hudson"); solo.clickOnImageButton(0); boolean expected = true; boolean actual = solo.searchText("(http://hudson:8080)"); assertEquals("Can't add new Hudson Instance", expected, actual); }

10. Roboelectric public class MyActivityTest { private Activity activity; private Button pressMeButton; private TextView results; @Before public void setUp() throws Exception { activity = new MyActivity(); activity.onCreate(null); pressMeButton = (Button) activity.findViewById(R.id.press_me_button); results = (TextView) activity.findViewById(R.id.results_text_view); } @Test public void shouldUpdateResultsWhenButtonIsClicked() throws Exception { pressMeButton.performClick(); String resultsText = results.getText().toString(); assertThat(resultsText, equalTo("Testing Android Rocks!")); } } Running unit tests on virtual machine – instead of running the tests on emulator or real device

11. Cucumber Scenario based testing implemented in Ruby Use cases are described in natural language where keywords are mapped to Android activities Given My ”TestExample" app is running And I wait for "Do Androids Dream of Electric Sheep?" to appear Then take picture Then I press the menu key Then take picture And I select ”Options" from the menu Then take picture And I wait for "Enter text into input field" to appear And I select "Enter text into input field" And I wait for "Save new configuration" to appear Then take picture

12. Running tests on Android InstrumentationTestRunner – the primary plumbing for running tests on Android Android Debug Bridge (ADB) – connects the devices and manages all debug information flow Asserts – rich validation capabilities enabled by JUnit. Helps validating test results, system outputs or practically anything. Mock objects – methods for creating mock system objects such as content, service and intent.

13. Testing for device compliancy 1. Test only on real devices 2. Select at least one phone with each major Android release 3. Choose different form factors: Side sliders, tablets, small resolutions , large resolutions 4. Integrate test execution to your development process – catch failures early in development and notice regressions immediately

14. Some best practices 1. Application developers should create the test cases at the same time when they are writing the code 2. All test cases should be stored in version control – together with the source code 3. Use Continuous Integration and run the tests every time the code is changed 4. Avoid using emulators and rooted devices 5. Publish the test results to whole organization – real time

15. Typical failures 1. Application installation fails 2. Application crashes during execution 3. Scaling/Layout problems 4. Application hangs if some resource is not available 5. Problems in landscape/portrait mode

16. Our solution for multi device testing Testdroid Recorder An Eclipse plugin that automates the manual tasks of writing tests for Android applications Testdroid Server A server software that manages all aspects of automatically running tests on multiple Android devices simultaneously.

17. Demo (Video)

18. Questions?

19. Thank you!