Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automating the Gaps of Unit Testing Mobile Apps

2,657 views

Published on

Apple has provided an orchestra of instruments for developers to use to fine tune their apps and create the best user experience possible. At compile time, within the simulator, on the device and in the field. Apple offers some solid advice to offer developers on how to make their apps the best they could possibly be. This session will look at the series of tools available from using static analyzers, to knowing when to use the simulator, and when using the device is required to collect the data necessary to understand just how an application is performing. It will also show how to gather information from testers that are testing the app live in out in real world scenarios rather then just in the team room.

Published in: Mobile
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/DMVvt ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/DMVvt ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Automating the Gaps of Unit Testing Mobile Apps

  1. 1. Automating the Gaps of Unit Testing Mobile Apps Effectively Utilizing Xcode’s Instruments
  2. 2. @ggeoffreggeoffre.com 20+ years of experience in the development, marketing, sales, and leadership of computer software development A C T S
  3. 3. Preparing your QA team for mobile testing http://www.slideshare.net/ggeoffre
  4. 4. Where did the term "bug" come from? 66 years ago on September 9th 1947, operators of the Mark II Aiken Relay Computer being tested at Harvard University, found something curious trapped between points at Relay #70, Panel F.
  5. 5. Bugs not Defects!
  6. 6. Bugs not Defects!
  7. 7. Bugs not Defects!
  8. 8. Bugs not Defects!
  9. 9. Bugs not Defects! Outside the scope of most acceptance or unit testing Tend to design around them not to them Defensive programming techniques employed More likely hardware based than software based Environment plays a major role discovering them Magnified greatly with mobile based development
  10. 10. Controlled Environments
  11. 11. Today’s Topics Reference Counting Battery Usage Network Monitoring Communicating Build and Version Numbers Xcode Server Initial Setup and Bot Creation Test Deployment Options to Devices Crash and User Analytics Reporting DevOps UITesting Instruments Automation Instrument Calabash Appium UIAutomation XCTest
  12. 12. Xcode ▶︎Open Developer Tool ▶︎Instruments
  13. 13. Reference Counting
  14. 14. [Objective-C] alloc/init retain copy release autorelease
  15. 15. Properties and @IBOutlets atomic nonatomic strong weak retain assign unsafe_unretained copy readonly readwrite
  16. 16. Automatic Reference Counting Edit => Convert => To Objective-C ARC Project => Build Settings
  17. 17. Swift When to use ? and when to use ! When to use “if let”? When to use as? and when to use as
  18. 18. WWDC 2015 Optimizing Swift Performance https://developer.apple.com/videos/play/wwdc2015-409/
  19. 19. WWDC 2016 Understanding Swift Performance https://developer.apple.com/videos/play/wwdc2016/416/
  20. 20. Xcode 8 InstrumentsI
  21. 21. Leaks The Leaks profiling template uses the Allocations and Leaks instruments to measure general memory usage in your app and check for leaks— memory that has been allocated to objects that are no longer referenced and reachable. https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingLeakedMemory.html iOS Developer Library - Instruments User Guide
  22. 22. Leaks https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingLeakedMemory.html iOS Developer Library - Instruments User Guide
  23. 23. Allocations The Allocations profiling template uses the Allocations and VM Tracker instruments to measure general and virtual memory usage in your app. However, to track down abandoned memory that’s been allocated but isn’t needed again, you’ll focus strictly on the Allocations instrument. This instrument measures heap memory usage and tracks allocations, including specific object allocations by class. https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingAbandonedMemory.html iOS Developer Library - Instruments User Guide
  24. 24. Allocations https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingAbandonedMemory.html iOS Developer Library - Instruments User Guide
  25. 25. Zombies The Zombies profiling template uses the Allocations instrument to measure general memory usage in your app, with a focus on the detection of overreleased “zombie” objects—that is, objects that are called after they’ve been released and no longer exist. https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/EradicatingZombies.html iOS Developer Library - Instruments User Guide
  26. 26. Zombies https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/EradicatingZombies.html iOS Developer Library - Instruments User Guide
  27. 27. Energy Savvy Users Battery Life is very important to users iOS Now Displays Apps in order of Battery Usage Be sure to test in Low Power Mode
  28. 28. Energy Gauge
  29. 29. Energy Diagnostics Settings > Developer > Instruments Enable Energy Recording Start Recording Xcode > Instruments New Energy Log File > Import Legged Data from Device
  30. 30. Energy Diagnostics
  31. 31. WWDC 2015 Debugging Energy Issues https://developer.apple.com/videos/play/wwdc2015-708/
  32. 32. Data Savvy Users Data caps are real iOS displays cellular data usage and allows users to turn on/off apps Be sure to test with Network Conditioner
  33. 33. Network Conditioner Settings > Developer > Network Link Conditioner Enable Choose a Profile
  34. 34. Network Instrument
  35. 35. Network Monitoring
  36. 36. Network Monitoring Use iTunes to find your UDID UDID
  37. 37. Network Monitoring Use Xcode Devices Window to find your UDID UDID
  38. 38. Network Monitoring rvictl -s <UDID> ifconfig rvi0 tcpdump -s 0 -A 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)' ... rvictl -x <UDID>
  39. 39. HTTPS Server Trust Evaluation Technical Note TN2232 App Transport Security Exceptions <key>NSAppTransportSecurity</key> <dict> <key>NSExceptionDomains</key> <dict> <key>yourdomain.com</key> <dict> <!--Include to allow subdomains--> <key>NSIncludesSubdomains</key> <true/> <!--Include to allow HTTP requests--> <key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key> <true/> <!--Include to specify minimum TLS version--> <key>NSTemporaryExceptionMinimumTLSVersion</key> <string>TLSv1.1</string> </dict> </dict> </dict>
  40. 40. macOS Sierra Apps WireShark tastycocoabytes.comwireshark.org Cocoa Packet Analyzer
  41. 41. WWDC 2015 Your App and Next Generation Networks https://developer.apple.com/videos/play/wwdc2015-719/
  42. 42. Asynchronous Programming Concurrent code execution on multicore hardware Grand Centeral Dispatch Blocks Closures UIView Animations NSNotifications Local and Remote Push Notifications
  43. 43. Blocks [Objective-C] dispatch_async(/* a queue /*, ^{ /* your code here */ }); dispatch_get_main_queue() dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)
  44. 44. dispatch_async(/* a queue /*, { /* your code here */ }) Closures Swift dispatch_get_main_queue() dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)
  45. 45. func perform_on_main(block: dispatch_block_t?) { dispatch_async( dispatch_get_main_queue(), block) } func perform_in_background(block: dispatch_block_t?) { dispatch_async( dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), block) } Closures Swift
  46. 46. perform_on_main { /* your code here */ } perform_in_background { /* your code here */ } Closures Swift
  47. 47. [UIView animateWithDuration:1.0 delay: 1.0 options: UIViewAnimationOptionCurveEaseIn animations:^{ /* your code here */ } completion:^(BOOL finished){ /* your code here */ }]; [Objective-C]UIView Animation
  48. 48. UIView Animation UIView.animateWithDuration(0.7, delay: 1.0, options: .CurveEaseIn, animations: { /* your code here */ }, completion: { finished in /* your code here */ }) Swift
  49. 49. NSNotifications [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(someMethod:) name:@"SOME_NOTIFICATION" object:nil ]; [[NSNotificationCenter defaultCenter] postNotificationName:@"SOME_NOTIFICATION" object:self ]; [[NSNotificationCenter defaultCenter] removeObserver:self ]; [Objective-C]
  50. 50. NSNotifications NSNotificationCenter.defaultCenter() .addObserver(self, selector:"someMethod", name:"SOME_NOTIFICATION", object:nil) NSNotificationCenter.defaultCenter() .postNotificationName("SOME_NOTIFICATION", object: self) NSNotificationCenter.defaultCenter() .removeObserver(self) Swift
  51. 51. WWDC 2015 Building Responsive and Efficient Apps with GCD https://developer.apple.com/videos/play/wwdc2015-718/
  52. 52. WWDC 2015 Designing with Animation https://developer.apple.com/videos/play/wwdc2015-803/
  53. 53. WWDC 2011 Using Local And Push Notifications https://developer.apple.com/videos/play/wwdc2011-517/
  54. 54. WWDC 2015 What's New in Notifications https://developer.apple.com/videos/play/wwdc2015-720/
  55. 55. WWDC 2016 Advances in UIKit Animations and Transitions https://developer.apple.com/videos/play/wwdc2016/216/
  56. 56. WWDC 2016 Thread Sanitizer and Static Analysis https://developer.apple.com/videos/play/wwdc2016/412/
  57. 57. Testing the User Interface
  58. 58. iOS Human Interface Guidelines https://developer.apple.com/ios/human-interface- guidelines/overview/design-principles/
  59. 59. UIKit Catalog (iOS) https://developer.apple.com/library/content/samplecode/UICatalo g
  60. 60. Accessibility for Developers https://developer.apple.com/accessibility/
  61. 61. Accessibility Audit Xcode => Open Developer Tool => Accessibility Inspector
  62. 62. Accessibility Attributes name Derived from the accessibility label. value The current value of the control, for example, the text in a text field. elements Any child elements contained within the current element, for example, the cells in a table view. parent The element that contains the current element. The name property is one of four properties of these elements that can be very useful in your test scripts. Select and Set
  63. 63. Accessibility Inspector Xcode => Open Developer Tool => Accessibility Inspector Accessibility Label
  64. 64. WWDC 2015 iOS Accessibility https://developer.apple.com/videos/play/wwdc2015-201/
  65. 65. Auditing Your Apps for Accessibility https://developer.apple.com/videos/play/wwdc2016/407/ WWDC 2016
  66. 66. AUTOMATION Behavior Driven Development Acceptance Tests JavaScript Tool Based (Command Line Execution) Instruments
  67. 67. AUTOMATION Install Xcode from the App Store https://itunes.apple.com/us/app/xcode/id497799835?mt=12 Xcode => Open Developer Tool => Instruments Choose “Automation” Select the target app to test with Record/Run
  68. 68. AUTOMATION Accessibility Label
  69. 69. TuneupJS.org Tune-up is a collection of JavaScript utilities that builds upon and improves the UIAutomation library provided by Apple for testing iOS applications via Instruments (get it? "tune-up"? Instruments? get it?). http://www.tuneupjs.org
  70. 70. T XCUIApplication() Behavior Driven Development Acceptance Tests Swift or Objective-C (XCTest) Tool Based, Project Test Target, (Command Line Execution) Xcode, Xcode Server Bot
  71. 71. T XCUIApplication() Install Xcode from the App Store https://itunes.apple.com/us/app/xcode/id497799835?mt=12 Create a new Project Product => Test Record
  72. 72. T XCUIApplication() let app = XCUIApplication() let someTextField = app.textFields["MyCustomLabel"] Find Things someTextField.tap() someTextField.typeText("qwerty") Do Some Stuff let expectaion = self.expectationWithDescription("Something") expectaion.fulfill() self.waitForExpectationsWithTimeout(5.0, handler: nil) Set Expectations
  73. 73. T XCUIApplication() Xcode
  74. 74. T XCUIApplication() xcodebuild test -project SampleProject.xcodeproj -scheme SampleProject -destination 'platform=iOS Simulator, name=iPhone 6s' Command Line
  75. 75. Behavior Driven Development Acceptance Tests Ruby/Cucumber/Bash Command Line Scripting RubyMine (optional)
  76. 76. Clone the iso-smoke-test-app repo… https://github.com/calabash/ios-smoke-test-app Install the calabash sandbox… (setup Ruby) https://github.com/calabash/install Build a .app for the Simulator… Run your first cucumber test:
  77. 77. Scenario: Touch OK button on alert When I touch the show alert button Then I see an alert Then I see the "Smoke Test!" alert And I wait for all animations to stop And I can dismiss the alert with the OK button When(/^I touch the show alert button$/) do wait_for_element_exists("view marked:'show alert'") touch("view marked:'show alert'") end Features Steps Ruby Gherkin
  78. 78. Behavior Driven Development Acceptance Tests Earl Grey
  79. 79. WWDC 2015 UI Testing in Xcode https://developer.apple.com/videos/play/wwdc2015-406/
  80. 80. Managing Simulators Visible and Meaningful Version Numbers Xcode Server Bots for CI iTunes Connect Crash Analytics Flurry User Analytics and Reports Fabric Crash Analytics Support HockeyApp QA and Beta Testing Some DevOps Tips
  81. 81. Manage Simulators Window => Devices
  82. 82. Add a new Run Script Phase Reset All Simulators osascript -e 'tell application "iOS Simulator" to quit' osascript -e 'tell application "Simulator" to quit' xcrun simctl erase all Project => Target => Build Phases
  83. 83. Run Script Git Version Info Build Version Number buildVersion=$(/usr/libexec/PlistBuddy -c "Print CFBundleShortVersionString” “${PROJECT_DIR}/${INFOPLIST_FILE}") buildNumber=$(git rev-parse --verify HEAD | cut -c1-5) buildString=$buildVersion' ('$buildNumber')' /usr/libexec/PlistBuddy -c "Set PreferenceSpecifiers:1:DefaultValue $buildString” "${SRCROOT}/Settings.bundle/Root.plist" /usr/libexec/PlistBuddy -c "Set :CFBundleVersion $buildNumber" "${PROJECT_DIR}/${INFOPLIST_FILE}"
  84. 84. Build Version Number
  85. 85. File => New => Settings Bundle Build Version Number
  86. 86. Settings Bundle Root.strings Build Version Number /* A single strings file, whose title is specified in your preferences schema. The strings files provide the localized content to display to the user for each of your preferences. */ "Group" = "SampleProject"; "Name" = "Version";
  87. 87. Settings Bundle Root.plist Build Version Number <?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"><dict> <key>PreferenceSpecifiers</key> <array> <dict> <key>Title</key> <string>About</string> <key>Type</key> <string>PSGroupSpecifier</string> </dict> <dict> <key>DefaultValue</key> <string>1.0</string> <key>Key</key> <string>version_preference</string> <key>Title</key> <string>Version</string> <key>Type</key> <string>PSTitleValueSpecifier</string> </dict> </array> <key>StringsTable</key> <string>Root</string> </dict></plist>
  88. 88. Install OS X Server and Xcode on the target Mac Enable the Xcode service (administrator account) Add the remote repository for the project Create a “BOT” to build and test the project View the results in the browser Setup Xcode Server
  89. 89. Enable Xcode Server
  90. 90. Create Bot for Project
  91. 91. WebView Screen Saver GitHub / liquidx / webviewscreensaver
  92. 92. WWDC 2016 Advanced Testing and Continuous Integration https://developer.apple.com/videos/play/wwdc2016/409/
  93. 93. Crash Analytics Technical Note TN2151 Understanding and Analyzing Application Crash Reports https://developer.apple.com/library/content/technotes/tn2151/_index.html App Distribution Guide - Analyzing Crash Reports https://developer.apple.com/library/content/documentation/IDEs/Conceptual/ AppDistributionGuide/AnalyzingCrashReports/AnalyzingCrashReports.html
  94. 94. WWDC 2015 Getting the Most out of App Analytics https://developer.apple.com/videos/play/wwdc2015/303/
  95. 95. Flurry Measure, track and analyze app performance, user acquisition and activity with Flurry Analytics. Get a deep understanding of app performance metrics and everything your users are doing. Create a Yahoo account and get your API Keys Add the Flurry framework to your project Integrate Flurry Analytics into your App https://developer.yahoo.com/analytics/
  96. 96. Fabric hooks deeply into your existing build process and automatically upload your dSYMs so you don’t have to. Now you can sleep well, knowing you’ll get the crash reports you need to keep your app stable and your users happy. Create and confirming your Fabric account Select the project you want to integrate Fabric into Add the Fabric framework to your project Fabric https://developer.apple.com/testflight/
  97. 97. Distribute beta versions, collect live crash reports, get feedback from real users and analyze test coverage. Bring DevOps to your apps by integrating with your existing build and work item management. Create an Account to Manage Teams and Apps Uploading a Build for an App Set Up and Invite Testers HockeyApp http://hockeyapp.net/features/
  98. 98. TestFlight Beta Testing makes it easy to invite users to test your iOS, watchOS, and tvOS apps before you release them on the App Store. You can invite up to 2,000 testers using just their email address. Create an App Store iTunes Connect Record Uploading a Build for an App Set Up and Invite Testers TestFlight https://developer.apple.com/testflight/
  99. 99. ggeoffre
  100. 100. ggeoffre slideshare.net http://www.slideshare.net/ggeoffre

×