Qa extreme 2011 eran kinsbruner

Trends and accentuations around
mobile application testing

eranki@matrix.co.il

Agenda
- Testing challenges
- Existing mobile Operating systems
- Mobile applications testing
- Porting
- Testing tools
- Real life examples
- Trends and accentuations in the mobile world
- Q&A

Testing Challenges
Device battery consumption
Device memory constrains
Screen size/resolution, Touch screen
Form factors (Slider, Clamshell, Block, etc.)
Performance of the device
Fragmentation among device OS’s
Price of defects

Testing Challenges - Continue
Security element in device testing (Apple UDID’s,
prompts for unsigned apps etc.)
Incoming events during testing, Coex testing
(Combinations, complex scenario’s, launch camera etc.)
Signal strength
Back/Exit buttons behavior among devices
Installation method (OTA, Serial cable, SDK’s)
Device Access point configuration
Various OS versions compliance
Various languages/locales

Mobile devices operating systems – Trends
(Gartner, September 2010)

Existing mobile Operating systems

• Symbian (Nokia)
• iOS (Apple)
• Android (Google)
• Blackberry OS (RIM)
• Windows Phone 7 (Microsoft)
• J2ME proprietary OS (SE/Samsung/LG) ?
• Palm/WebOS (Palm Pre) ?
http://www.calcalist.co.il/internet/articles/0,7340,L
-3519738,00.html)

Symbian OS
• Symbian OS was originally developed by Symbian Ltd, which in 2008 was acquired
by Nokia.

• Symbian OS had almost no UI Layer, Most UI design was made by third parties
– S60 by Nokia
– UIQ by SonyEricsson
– MOAP by NTT DoCoMo

• In 2008 with Nokia acquiring Symbian Ltd. the Symbian Foundation was created and
the three UIs were contributed to it in order to create a Unified Symbian Platform.

Symbian OS - Continue
• Today two streams of Symbian UIs are available.
– Symbian^3 is used by Nokia’s High-end smartphones.

– S60 is used in Mid-Range smartphones.

• It contains associated libraries, user interface, frameworks and reference implementations of
common tools.
• Symbian runs on ARM processors (x86 unofficial).
• Symbian applications are packed in SISX files and requires Symbian Signed certificate to be
commercially deployed on a handset.
• SEMC recently announced that it is deserting Symbian OS to switch to the more popular OS’s –
Android & Windows Phone 7
• Nokia recently announced that it is also deserting Symbian and is about to move to Windows
Phone 7/8 OS for its future devices.
• Note: NTT DocoMo and Fujitsu announced a new Tablet/Smartphone running Windows 7 OS
(Tablet) and Symbian (Mobile) on one platform (Details)

Android OS
• Android is a mobile operating system developed by Google and is based upon a
modified version of the Linux kernel. It was initially developed by Android Inc. (a firm
purchased by Google)

• Its OS versions are usually incremented alphabetically and named upon famous
cookies/cakes
• 1.5 cupcake
• 1.6 donut
• 2.0 éclair
• 2.2 Froyo
• 2.3 Gingerbread
• 3.0 Honeycomb
• ? IceCream

• What is ADB ?

Android – Screen resolution method

• In Android, Google developed a new way of handling different screen
resolutions, in order to save porting time of applications among various of
Android phones
• Basically Android OS divides its screen size to 4 different sizes and 4 different densities and
the platform transparently handles the rendering of the layouts at the actual screen size
• Small, Normal, Large and Extra Large
• ldpi (low), mdpi (medium), hdpi (high), and xhdpi (extra high)

• Detail explanation of how it’s done can be found in a post at TACT/Blogix
web page: http://qa.blogix.co.il/2011/04/16/%D7%90%D7%99%D7%9A-
%D7%9E%D7%A2%D7%A8%D7%9B%D7%AA-
%D7%94%D7%94%D7%A4%D7%A2%D7%9C%D7%94-
%D7%90%D7%A0%D7%93%D7%A8%D7%95%D7%90%D7%99%D7%93-
%D7%93%D7%95%D7%90%D7%92%D7%AA-
%D7%9C%D7%94%D7%A6%D7%92%D7%94/
• Top rated Android apps for 2011 - http://www.web-designers-directory.org/articles/top-
rated-android-applications-for-2011-20.html

Blackberry OS
• BlackBerry OS is a proprietary mobile operating system, developed by
Research In Motion for its BlackBerry line of smart phone handheld devices.
• BlackBerry applications are developed in Java with RIM’s SDK
• The operating system provides multi-tasking, server-side push service and
supports specialized input devices:
• trackwheel (7100i, OS 4.1.0)
• trackball (9630 Tour, OS 5.0)
• trackpad (9780 Bold, OS 6.0)
• touchscreen (9800 Torch, OS 6.0).
• BlackBerry Various Emulators for testing - Download

Apple iOS
• iOS formerly known as iPhone OS is Apple’s proprietary mobile operating
system. Developed originally for the iPhone, it has since been shipped on the
iPod Touch and iPad as well.
• iOS 1.0 – iPhone
• iOS 2.0 – iPhone 3G
• iOS 3.0 – iPhone 3Gs
• iOS 4.0 – iPhone 4
• iOS 4.3.3 was released on 5-May, 2011 and is the version used on latest
iPhone 4 devices
• MAC OS tools:
• Xcode, Interface Builder, iPhone Simulator

Apple iOS - Continue
• There are few free tools which can allow a developer to develop/test his application for iPhone:
• For Windows OS there are:
• AirPlay https://www.airplaysdk.com/downloadsall.php
• C:Airplay SDK4.4examplesgameskartz

• Titanium http://www.appcelerator.com/products/download/ (JavaScript developing for iPhone, Android)

• Mono Touch Develop Android and iPhone apps with C# on windows - not free (http://www.go-mono.com/mono-
downloads/download.html (http://mono-tools.com/download/)

• For MAC OS iPhone SDK (Xcode) provided by Apple with full development environment (C++ or
Objective C languages)

• Top 10 iPhone apps by TIME magazine:
http://www.time.com/time/specials/packages/completelist/0,29569,2044480,00.html

• Xcode Demo’s

• http://www.youtube.com/watch?v=miIBUq4mGSw

• http://www.youtube.com/watch?v=y5XYKl3iHDM&feature=relmfu

Windows Mobile
• Windows Mobile is a mobile operating system developed by Microsoft for use
in smart phones (non-touch) and PPC (touch) devices.
• WM features a suite of basic applications developed using the Microsoft
Windows API. It is designed to be somewhat similar to desktop versions of
Windows, feature-wise and aesthetically. Additionally, third-party software
can be purchased via the Windows Marketplace
• The OS is now obsolete and The Last version is "Windows Mobile 6.5.3“
(based on the Windows CE 5.2 kernel)

• Sample devices:
• HTC HD2
• Sony Ericsson XPERIA X2

Windows Phone 7
• Windows Phone 7 is the successor to Microsoft'Windows Mobile platform
s
(based on Windows CE 6.0 R3 kernel).
• Windows phone 7 is built from the ground up and features no backward
compatibility to the previous 6.5 version.
– Does require certification and signing of app to be publicly deployed
• Some of the features are
– Live Tiles,
– Social networking integration,
– multi-touch Support,
– internet explorer mobile,
– Office 2010 Mobile,
– Search & Maps using Bing,
– Xbox live gaming
– Metro UI L&F
(http://www.microsoft.com/windowsphone/en-us/default.aspx)

Windows Phone 7 – Dev. & Testing
Windows Phone 7 development tools are distributed free by Microsoft.
Important and up to date data is constantly updated at http://blogs.microsoft.co.il/blogs/msdn/

• Windows Phone Developer Tools http://www.microsoft.com/express/Downloads/#2010-Visual-Phone

• Expression Blend is a more user friendly application used to design only the User Interface
Experience

• There is a newly developed application market for windows phone 7 platform App Hub
http://create.msdn.com/en-US/
• Preliminary application market for Israel (Yalla Apps - http://www.yallaapps.com/ )

• A training kit for Windows Phone 7 new comers can be obtained from http://msdn.microsoft.com/en-
us/wp7trainingcourse.aspx

Zune Sync music and video for WP7 devices
**

• Note: The tools above require Windows 7 to be installed

Windows Phone 7 – Dev. & Testing Visual
Studio Express 2010

Windows Phone 7 – Expression Blend 4
Rapid Prototyping without writing code

Windows Phone 7 & SilverLight

• Short demo
http://isvis.com/work/silverlight/3d/tracking/
• Silverlight is the application development platform for Windows Phone 7.
• Silverlight for Windows Phone supports core Silverlight capabilities in
managed .NET code with XAML including:
– High quality video and audio using a wide range of codecs, DRM and IIS
Smooth Streaming
– Deep Zoom for enhanced reading and photo browsing experiences
– Vector and Bitmap Graphics and animation
• Note: In order to deploy your app on a real phone, the phone
must be registered within MSFT (Emulator is the preferred
preliminary way of testing)

Windows Phone 7 & SilverLight

• Silverlight can also access the unique capabilities of the phone including:
– Hardware acceleration for video and graphics
– Accelerometer for motion sensing
– Multi-touch
– Camera and microphone
– Location awareness
– Push notifications
– Native phone functionality
• Silverlight can also utilize the XNA Framework for Audio capture and playback,
Media Library Access, and even accessing Xbox LIVE.
• Currently Windows Phone 7 uses silverlight 3.0, soon silverlight 4.0 with additional
features will be released as part of the next MSFT SDK (MANGO)
• Tombstone Vs. Dormant (MANGO Release)

Windows Phone 7 Leading Phones

Samsung Focus - http://www.microsoft.com/windowsphone/en-
us/buy/7/phones.aspx?id=1536&id=1536#detail=1536
HTC Arrive - http://www.microsoft.com/windowsphone/en-
HTC HD7 - http://www.microsoft.com/windowsphone/en-
LG Quantum - http://www.microsoft.com/windowsphone/en-
HTC Surround - http://www.microsoft.com/windowsphone/en-
Dell Venue Pro - http://www.microsoft.com/windowsphone/en-

Mobile Application Testing - Methodologies

• The testing methodology mostly depends on the type of application.
• There are 3 main mobile applications:
1) Stand alone Taking into account the mobile phone limitations, such
applications shall be tested by Functional tests, Usability tests,
performance tests.
2) Distributed Testing for both client and server side alone and together
is required (e.g- Instant messaging application)

Mobile Application Testing - Methodologies

3) Web based In a Web-based application the client is an Internet browser,
and the main functionality is built on the server-side. Testing Web based
applications require the following type of testing:
• Functional testing
• Compatibility testing
• Content testing
• Performance testing
• Load testing
• Security testing
• Server-side testing (Web server, application
server)
• Database testing

Mobile Application Testing – Proposed strategy

• Do invest in a device emulator
– It’s a well instrumented testing environment, usually more than 1 skin
– Cost effective
– Bypasses real network
– Easy to debug
• Take advantage of a real remote device solution (Nokia, Device Anywhere,
Perfectomobile)
– “Hire” target device/s per hour to perform real device testing as needed
• Don’t completely avoid testing on real device on real network

Conclusion: Seek a solution which combines remote manual testing on real
device, emulated device testing and automation tests wherever possible.

Mobile Application Testing – Porting Concepts
1 of 3

Since there are variety of devices from each OEM, each has different
constraints (Memory, screen size, heap size, form factor, camera, touch
screen, installation path etc.) – it is a must to plan the application to be as
much portable as possible during the design phases.
Not taking into account the devices family and the above constrains will
require massive porting and long development cycles when starting the
deployment
Having configurable pre-compiled options which vary between devices
would ease the porting and enable to use the same code (Mostly) for all
devices, with only per device configuration options

2 of 3

Having more than one “gold” target device during development phase is also
an important requirement
Using automation solution as mentioned earlier and SDK’s is also an
advantage.
There are special application which can provide quite easily the list of
supported features per device (TasePhone Java application e.g.)
Using emulation environments per device is cheap and cost effective during
development
Taking into account the resources used by the application is important (there
are tools who can shrink the entire folder of resources into a small size file
which can be used by the application – BoomFs)

3 of 3
When a new or pre-launched phone should be supported by the application
developers there are few things which should be taken into account:
Phone characteristics (Screen size, OS, memory, form factor, supported features
such as camera and others) Can be retrieved for most phones through GSM
Arena site
How close the phone is to the lead device within a family which is already
supported
If the phone represents a new family this might have implications on the porting
time in opposed to a new member to an existing family
Is there a way to upgrade phone OS to overcome firmware defects (Cables,
contact with the OEM etc.)
It would always be good to have at least 2 devices from this kind to be able to
reproduce defects, compare performance and firmwares.
How the application works in comparison to other apps on the device (Closing
method, events reaction and more)

Mobile Application Testing – Bug Trackers

• There are few free/cheap tools (Clients) which can be installed on your
mobile device and allow you to submit bugs on the fly as you test your
application
– JIRA Mobile client for Android Beta (and soon iPhone)
http://blogs.atlassian.com/jira/2010/05/jira-android-bug-tracking-mobile-phone-
device-20100520b.html
– FogBugs https://www.fogcreek.com/fogbugz/mobile/
• Supports iPhone, Android, Blackberry

Mobile Application Testing – Battery
Consumption
• As mentioned earlier, the battery of the mobile phone is limited and the application
developers must take into account the application battery usage, performance and
take relevant steps and optimizations in order to assure that the application
consumes the minimal battery resources as needed
– There are monitoring tools which can be used to check the battery usage (Not always
specific per application but this can be narrowed to the specific application vendor
needs)
For iPhone
Battery Life Pro http://itunes.apple.com/us/app/battery-life-pro-all-in-1/id373041141?mt=8
Battery Led

Mobile Application Testing – Battery
Consumption - Continue

• For Android OS there is e.g. the ‘Power Tutor’ tool
– http://gigaom.com/mobile/android-power-consumption-app/

• For windows Mobile
– http://www.freewarepocketpc.net/ppc-download-wr-tools-resinfo.html

Mobile Application Testing – Leading
automation tools

There are 2-3 main players in the device test automation world
with proven results and ROI
Perfectomobile
http://perfectomobile.com/?key=adwords2&gclid=CM_G0eWipaQCFR5
N5QodNUXJ6w
Remote Access -
http://perfectomobile.com/portal/cms/Products/Web_Access_To_Real_Handsets.html?k
ey=adwords2
Automation -
http://perfectomobile.com/portal/cms/Products/automated_testing.html?key=adwords2

Mobile Application Testing – PerfectoMobile

automation tools

Device Anywhere - http://www.deviceanywhere.com/mobile-application-
testing-overview.html

Mobile Application Testing – Device Anywhere

automation tools/SDK’s

• Experitest - http://experitest.com/
• Provides Emulator and real device automation solution (Record/replay and code generation)
• Supports the following OS:
• Android
• iOS
• Blackberry
• Windows Mobile
• Symbian
• Exports code to C#, Java, Python, QTP-VBScript, Perl

• Robotium - http://code.google.com/p/robotium/
• robust automatic black-box test cases for Android applications (Run your application as an
Android Junit test on Android SDK emulator)

Mobile Application Testing – Experitest

Mobile Application Testing – SDK’s &
Emulator’s

In order to test application developed for mobile, often the vendors would prefer to
use as many devices SDK’s and Emulators to get as close to reality L&F and
functionality before deploying first on device which has limited debugging capabilities
in opposed to the SDK’s
Few SDK’s which are free to use
NetBNeans IDE which has simple integration for most common OEM SDK’s (Motorola, Nokia, SEMC)
- http://netbeans.org
Nokia specific Emulation environment -
http://www.forum.nokia.com/Library/Tools_and_downloads/Archive.xhtml
SEMC specific Emulation environment
1) http://developer.sonyericsson.com/wportal/devworld/technology/android/developer-tools
2) http://developer.sonyericsson.com/wportal/devworld/technology/windows-mobile/xperia-x2
Motorola specific Emulation environment -
http://developer.motorola.com/docstools/sdks/archive/c975ADK/
Sun Java Wirelsss Toolkit (J2ME SDK for application developers) -
http://java.sun.com/products/sjwtoolkit/download-2_5_1.html

Emulator’s

J2ME
Sun Java Wireless SDK -
http://www.oracle.com/technetwork/java/download-135801.html
(Emulation/debugging environment)
NetBeans SDK – IDE with OEM SDK’s plug in integration for pre-device
testing and debugging
Mobile Junit by SEMC
Jbenchmark (http://www.jbenchmark.com/tools.jsp)
LG -
http://developer.lgmobile.com/lge.mdn.tnd.RetrieveDocAndTools.dev?object
Type=T#1
Samsung -
http://innovator.samsungmobile.com/down/cnts/toolSDK.detail.view.do?platf
ormId=1&cntsId=9460

Emulator’s

Symbian
Symbian signed pre-certification tools
(http://developer.symbian.org/wiki/index.php/Symbian_Signed_Tools)
Windows Mobile (I assume it will be EOL’d in the near future due to Windows
Phone 7 platform)
Test Complete (Allows connection from device to Microsoft
‘ActiveSync’/’Windows Mobile device center Synchronization’ SW).
http://www.automatedqa.com/products/testcomplete/testing-pda-apps/
Windows mobile emulators -
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=38c46aa8-
1dd7-426f-a913-4f370a65a582&DisplayLang=en

Real Life Experience overview

Driver and Firmware for WLAN/BT/GPS testing on mobile platform running
Android/Linux (IOP testing)

Android SDK – Use Android SDK to pre-test your application on a variety of
Emulators (Uses the latest open source file system)
Google CTS
ADB Shell

Mobile Trends and Forecast

Mobile Devices Begin to Replace Credit Cards
2011 looks like the year when, in some countries at least, using
your phone as a credit card for lower value purchases will
become a reality. Google recently announced that NFC (Near
Field Communication) technology will be supported in the next
release of Android – 2.3 or “Gingerbread”;
HTC Flyer phone announced to support NFC within a year
Social networking based devices (Recently HTC announced that
they launched social networking specific devices - ChaCha and
Salsa)

Mobile Trends and Forecast - Continue

Smartphones invading the enterprise (Security threats will be big factors in
determining which companies aggressively deploy smartphones)

Mobile OS battles will continue (Android will become the leading Smartphone
platform for at least the next four years, according to the latest data from IDC. The
analyst firm estimates that Android will account for 45 per cent of the global
Smartphone market by 2015, up from under 40 per cent today (30-Mar-11). RIM and
iOS will see more modest growth and a slight fall in overall market share to around 15
per cent

Smartphone and tablet convergence (growing number of Smartphone manufacturers
will feel pressured to introduce tablet computers, and will likely do so this year.
Offering one but not the other will become an intensifying threat to their businesses)

Mobile Trends and Forecast - Continue

Cloud in Mobile – Apple set to launch cloud-based music service
ahead of Google (In this model, people can upload and store all
their music to Apple servers, and launch it from any device or
share it with other users (Using an iTunes account)).
Apple cuts Amazon & Google by signing the music companies

Appendix A – Sample certifications

Symbian Signed certification – Mandatory for any S60 V3 application which is
intended for commercial launch https://www.symbiansigned.com/app/page
Java Verified Certification – Sun’s guidelines for Java/Java ME compliant
applications http://javaverified.com
Windows Mobile application signing - http://msdn.microsoft.com/en-
us/windowsmobile/dd569132.aspx & Windows Phone 7 Certification
JSR TCK tests – Specific JSR compatibility tests which are mandatory to any JSR
OEM implementer - https://jtharness.dev.java.net/ (open source test harness) – For a
complete list of JSR’s see
http://www.jcp.org/en/jsr/platform?listBy=1&listByType=platform
Java Device test framework – specific JSR test suite with functional tests (Provided
to reduce fragmentation and increase quality -
http://java.sun.com/products/javadevice/index.jsp & https://jdtf.dev.java.net)

052-2424314
ryonish@tact.co.il

Qa extreme 2011 eran kinsbruner

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (6)

Similar to Qa extreme 2011 eran kinsbruner

Similar to Qa extreme 2011 eran kinsbruner (20)

Recently uploaded

Recently uploaded (20)

Qa extreme 2011 eran kinsbruner