1. iOS and Android Apps automation
By,
Sridhar Ramakrishnan
Life long travel geek @ Hotwire Inc.
2. Agenda
• How is Native app testing is different from Web testing
• Kind of Native app bugs
• Test automation
• Different framework for native apps automation
• What is appium & how it works
• Android & iOS frameworks – Espresso & Xcode UI testing
• Sample test code
• Demo
• Code Coverage
• Our CI process
• Screenshot testing – Image comparison tests
• Tracking
• So why we are doing all this??
3. How is Native app testing is different
from Web testing
• Usability & interaction
• Faster & more reactive
• Involves Gestures
• No unnecessary content
• Data Entry Keyboard Types
• Numeric vs. Alphanumeric
• Phone Settings
• Changing Regions
• Changing Location Services
• Changing Notifications
• Devices vs. Simulators
• Multiple Devices and Operating Systems
4. Bugs Types
• Crashes
• Missing data from the API
• Missing data in the app
• Wrong keyboard displayed
• Rendering differences across devices
• Views overlaying one another
• Views are getting locked up
• Gesture related – swipe, double tap
5. More Bugs ?
• Limited manual regression
testing = Spot check ; less
boring
• More manual exploratory
testing = More bugs;
“Simultaneously learning about
the system while designing and
executing tests, using feedback
from the last test to inform the
next.”
- Elisabeth Hendrickson
7. Frameworks comparison
Frameworks for
comparison
Test the
actual
app
Many
language
Standard API
( Webdriver)
Open
source &
active BDD iOS Android Hybrid
Firefox
OS
Appium
Xcode UI
Automation
calabash-iOS
Frank
ios-driver
KIF
Robotium
Espresso
selendroid
8. Test Automation @ Hotwire
• Test Framework = Hotwire BDD
• Tools =
Xcode UI testing
• API = Spoofer (mock) vs API calls
10. Appium Architecture
Test Script (
Java )
Webdriver
(Seleium Java)
Apple
Instruments
(Instruments.js)
iPhone / iPad
[Simulator /
Device]
APP
Android
[Simulator /
Device]
APP
Android (adb,
UIAutomator
(AppiumBootstr
ap.jar …)
FireFox OS /
Hybrid Apps
Firefox OS /
Web browser
BDD
(Cucumber,
Webdriver,
Spring, maven)
CI – Jenkins (Build xcode, start Appium server, mvn install
Appium server
(Node.js) JSJSON
11. BDD – Cucumber & Gherkin
Feature: Hotel Search and Book
Scenario: Search and book hotel as Sign In user
Given I'm searching for hotel and attempt to book
When I book a hotel as valid signed in user
Then I should see confirmation that my hotel is booked
12. BDD – Cucumber @tags
@iOS @Android @SMOKE
Feature: Hotel Search and Book
Scenario: Search and book hotel as Sign In user
Given I'm searching for hotel and attempt to book
When I book a hotel as valid signed in user
Then I should see confirmation that my hotel is booked
13. BDD – StepDefs
//Spring context file
@ContextConfiguration("classpath:cucumber.xml")
//hotel object gets input from Spring
@Autowired
private Hotel hotel;
@Given("^I'm searching for hotel and attempt to book$")
public void search() {
hotel.search();
}
14. Java code
public interface
Hotel {
void search();
}
// iOS
public class IosHotelImpl implements Hotel {
@Override
Public void search() {
IosSearchScreen iosSearchScreenObj = new
IosSearchScreen(getWebDriver());
}
// Android
public class AndroidHotelImpl implements Hotel {
@Override
Public void search() {
AndroidSearchScreen androidSearchScreenObj =
new AndroidSearchScreen(getWebDriver());
}
15. Screen Objects - iOS
// iOS Screen Object
public class iOSSearchScreen extends AbstractScreen {
@FindBy(name = ”Search")
private WebElement searchBtn;
public void search() {
searchBtn.click();
}
}
16. Screen Objects - Android
// Android Screen object
public class AndroidSearchScreen extends
AbstractScreen {
@FindBy(id= ”Search")
private WebElement searchBtn;
public void search() {
searchBtn.click();
}
}
17. Screen Objects
public class AbstractScreen {
private final WebDriver webdriver;
public AbstractScreen(WebDriver webdriver) {
PageFactory.initElements(webdriver, this);
this.webdriver = webdriver;
}
protected WebDriver getWebDriver() {
return webdriver;
}
}
18. BDD – StepDefs
//Spring context file
@ContextConfiguration("classpath:cucumber.xml")
//hotel object assigned by Spring
@Autowired
private Hotel hotel;
@Given("^I'm searching for hotel and attempt to book$")
public void search() {
hotel.search();
}
19. Dependency Injection
using Spring Framework
Context file - Cucumber.xml
<context:component-scan base-package="com.test" />
<import resource="cucumber-${mobileOS}.xml"/>
20. Dependency Injection
using Spring Framework
Cucumber-ios.xml
<bean id="hotel" class="com.test.ios.hotel.IosHotelImpl">
</bean>
<bean id="desiredCapabilities"
class="org.openqa.selenium.remote.DesiredCapabilities">
21. Dependency Injection
using Spring Framework
Cucumber-android.xml
<bean id="hotel"
class="com.test.android.hotel.AndroidHotelImpl">
</bean>
<bean id="desiredCapabilities"
class="org.openqa.selenium.remote.DesiredCapabilities">
22. Managing Dependency
3rd Party Jars
Cucumber
Spring
Reporting
Junit / testng
…
Tests Jars
Features
StepDefs
iOS-Screen-Objects
Android-Screen-Objects
…
23. Hotwire Apps BDD architecture
Screen Objects
(Objects representing screen /
fragment)
Appium server
(Node.js)
Apple
Instruments
with
UI Automation
iPhone / iPad
[Simulator /
Device]
APP
JSON
JS
31. Espresso
• Espresso is an open source Android test automation
framework from Google.
• Espresso is API for UI Testing on Android.
• Espresso core API is small, predictable and easy to
learn.
• Espresso is a thin layer on top of Android
Instrumentation that makes it easy to write reliable UI
tests.
• It supports all Android versions.
• Tests run optimally fast as there is no need for any
sleeps (tests run on same millisecond when the app
becomes idle)
32. Espresso: Code Example
public class EspressoTest extends ActivityInstrumentationTestCase2<HomeScreenActivity> {
public EspressoTest() {
super(HomeScreenActivity.class);
}
@Override
public void setUp() throws Exception {
super.setUp();
getActivity();
}
@Test
public void testSearchHotel() {
//finding a view with onView and perfoming an action
onView(withId(com.hotwire.hotels.R.id.current_location_edittext))
.perform(click());
onView(withId(com.hotwire.hotels.R.id.destination)).perform(typeText("San Francison"));
onView(withId(com.hotwire.hotels.R.id.search_hotels_button)).perform(click());
}
}
34. Xcode UI testing
• Available from Xcode 7
• New feature added to XCTest framework
• UI recording
• Ease to locate elements
• Run tests on multiple iOS versions
• Integrates with Xcode server
• Code coverage
35. But why? We already have Appium
- Faster execution time (1m45sec vs 5m45sec)
- Faster to write too (Record feature)
- No Appium/3rd party dependencies
- Free code coverage reports
- Handles wait times implicitly
- Snapshot of failed tests
- Consolidated reports
36. UI testing : Code Example
import XCTest
class HotwireUITests: XCTestCase {
override func setUp() {
super.setUp()
continueAfterFailure = false
XCUIApplication().launch()
}
func testIPadHotel() {
//HomeScreen verification
let homeScreen = HomeScreen()
homeScreen.verifyHomeScreenElements()
//Hotel Search with Signed In User
homeScreen.navigateToHotel()
//Hotel Farefinder verification
let hotelFF = HotelFareFinder()
hotelFF.verifyFareFinderElements()
hotelFF.findHotel()
}
import Foundation
import XCTest
class CarBookingScreen {
let elementsQuery =
XCUIApplication().scrollViews.otherElements
let toolbarsQuery = XCUIApplication().toolbars
func goToSignIn(){
elementsQuery.buttons["SIGN IN"].tap()
}
func completeBooking(){
enterCCV()
elementsQuery.switches["AgeRestriction"].tap()
elementsQuery.buttons["Book Now"].tap()
}
}
40. public class Main {
private String s;
public static void main(String[] args){
//some code goes here
if (something != null) {
//body
}
}
}
public class Main {
private String s; //magic infused
public static void main(String[] args){
//report that method was executed
//some code goes here
if (something != null) {
//report that condition was met
//body
}
}
}
Original class Instrumented class
Code coverage – Android using Emma
41. iOS / Android CI
Goal
To provide rapid feedback so that if a bug is introduced into the code base, it can
be identified and fixed as soon as possible.
Jenkins
We are using Jenkins for all our CI runs at the moment, triggered on each Git
check-in
iOS / Android CI Pipelines
• Master Pipeline
• Develop Pipeline
• Release Pipeline
• Feature Pipeline
Polling
Nodes/hosts from an iOS and Android host-cluster, polled for each leg of the
pipeline execution. Artifacts are zipped and transferred between jobs.
• Less wait time for resources
• Parallel job execution
42. iOS CI
Build * Unit Tests * iPad Smoke Car/Hotel, iPhone Smoke Car/Hotel * Regression
----- Meeting Notes (4/8/14 18:47) -----
Spring provides many features.
One major one is DI.
Easy to implement and integrate with this framework
----- Meeting Notes (4/8/14 18:47) -----
ios implementation
Also any user specifc data like user input for user creation
----- Meeting Notes (4/8/14 18:49) -----
Lets look at what is in the HotelImpl.java class
Simple script to generate HTML reports of the code coverage. This HTML report is used in Jenkins to show coverage report with every build.
What is Continuous Integration (CI)?
Isolated changes made by developers are continuously built, tested and reported as and when they are checked in as part of the codebase.