Developers have embraced Continuous Integration for years and it has proven their value for accelerating software production for Web environments. However, for mobile developers, it’s been a slow road to adopting many of these same practices. In this webinar, Kevin Rohling (Emberlight, Ship.io) and Kristian Meier (Sauce Labs) will cover best practices in implementing a mobile CI system and demonstrate how you can easily build, test and deploy mobile apps.
11. WHY DO I USE CI?
• No Touch Configuration
• Automated OTA Distribution
12. WHY DO I USE CI?
• No Touch Configuration
• Automated OTA Distribution
• Code Validation: Automated Builds and Tests
13. ONE SIZE DOES NOT FIT ALL
Business Drivers Engineering Challenges
Early Startup
Initial Traction,
Funding
Stability, UX
Consulting Firm
Client Status Updates,
Deadlines
Ad Hoc Distribution, Unit Testing
Large Product Team
Retention, Growth,
Revenue
Automation, Communication
PROCESS
17. MOBILE IS HARDER
• Infrastructure
• Compilation/Code Signing
• Testing
• Deployment
18. SIMULATORS/EMULATORS vs DEVICES
SIMULATORS:
• Used by iOS, included w/ Xcode
• Execute i386 instruction set, not ARM
• Very fast compared to Emulators
• Do not have access to certain hardware functions
• (GPS, Bluetooth Radio, Accelerometers, etc)
19. SIMULATORS/EMULATORS vs DEVICES
EMULATORS:
• Used by Android, included w/ Android SDKs
• Execute ARM (or native device instruction set)
• Very slow compared to Simulators
• Do not have access to certain hardware functions
• (GPS, Bluetooth Radio, Accelerometers, etc)
20. SIMULATORS/EMULATORS vs DEVICES
DEVICES:
• Reproduces the actual performance experienced by your users
• The only way to catch manufacturer/OEM bugs
• Very expensive to configure and maintain compared
• Full access to hardware functions:
• (GPS, Bluetooth Radio, Accelerometers, etc)
21. WHEN TO USE DEVICES vs SIMULATORS/EMULATORS?
• Simulators and Emulators are an excellent solution for running automated tests
during development. They are inexpensive and will reliably catch most problems.
• Physical Devices can be used on a lower frequency (i.e. pre-release, weekly,
daily, etc.). They are the only way to catch performance problems, test certain
hardware features and find OEM issues. In the least, devices should be used
before every release.
22. UNIT TESTING
According to a recent survey:
-77% of developers said app quality is “very important or mission critical”
-80% of mobile developers test their apps manually
-Only 13% of mobile developers use automated testing
23. UNIT vs FUNCTIONAL TESTING
• Unit Testing - testing small pieces of code
• Functional Testing - testing button clicks and UI interaction
26. • What language/framework do your developers know?
• Open Source/Community Support
• 3rd Party Framework requirements
UNIT/FUNCTIONAL TESTING FRAMEWORKS
WHICH ONE TO USE?
27. CI TOOLS/SERVICES FOR MOBILE
Good: Open Source, Lots of plugins
Bad: Self-Hosted, DIY solution
Jenkins
28. CI TOOLS/SERVICES FOR MOBILE
Good: Hosted solution, OS X support, Lots of plugins
Bad: Tedious setup process
Travis CI
29. CI TOOLS/SERVICES FOR MOBILE
Good: Hosted solution, Designed specifically for mobile, Easy setup
Bad: Less flexible than other solutions
Ship.io
30. CI TOOLS/SERVICES FOR MOBILE
Good: Integrated w/ Xcode, Apple-Supported
Bad: Self-Hosted, iOS Only
Xcode CI
Frequency for physical devices is very much based on budget
JUnit is the corresponding “Out Of The Box” solution for Android developers. Similiarly it is tightly integrated with Eclipse but can also be run from the command line using Ant. Test runs are quite slow because of the dependency on a running emulator.
Dubbed ‘Selenium for mobile apps”, Appium is an open source , cross-platform automation tool that enables you to write mobile tests in any language or framework. It interfaces directly with native automation frameworks so there’s no SDK to include, and the code you test is the same code you release.
Appium allows you to use many of your Selenium scripts and tests to speed testing for mobile devices meaning you can test more apps more quickly and get them to market faster.
When we wanted to enter into mobile testing. We found iOS auto – a one man project – and added our own effort, renamed it Appium and added Android support. We made this open source and now have seen huge adoption of this as big test framework.
Released in June 2014 as v1.o
718k downloads in first year,
Developed and supported by Sauce – 5 of top 6 contributors are sauce FTE
The customer’s CI server runs scripts which kick off tests that run on Sauce via our API.
We use ‘Sauce connect’ to bridge between our cloud and the customer’s applications using a connector piece using Sauce Connect. This works very well with Se as you just point your tests to the Sauce cloud to do the tests.
We believe we have the largest cloud for automated testing!
We are not just another platform but rather we produce nice test reports, screen shots, logs and videos – all these tools help speed your de-bugging time (such as quickly seeing that a screen didn’t render properly etc)
We integrated with the WC3 draft standards.
We have an enterprise SLA – 99.9% uptime
Develop a slide of benefits of parallelization in CI – Forrest! This is often lost in the discussion – helps get to the 100 VM conversation
There is a lot to learn so browse to our resources page to understand the latest about Sauce Labs.
We offer a free 14 day trial that gives you access to our full automation platform – up to 8:
8 concurrent VMs
90 hours of automated testing
Unlimited manual testing
Access to all reporting tools including
Screen shots
Videos
Logs and metadata