CI for iOS. CodeMash version.

1. CI for iOS:
do more while you sleep
Kevin Munc – @muncman
Method Up LLC
[MU]

2. CI?

3. For iOS?

5. Pain

6. Pain & Suffering

7. It can be done
• Start with the basics
• Xcode and the command line
• Version control and CI server
• Script it, run it, schedule it

8. Tip #1

9. Jenkins
• http://jenkins-ci.org/
• Mac installer or via homebrew
• Many alternatives
• Many plugins

11. Tip #1.1

12. How many Simulators does
it take to break a build?

13. A CI server gives you:

14. A CI server gives you:
Automation

15. A CI server gives you:
Automation
Feedback

16. A CI server gives you:
Automation
fu l!
er
ond
W
Feedback

17. A CI server gives you:
Automation nd )
i s
fu l! f m ap
er e o er n
ond ac tt
Pe be
W
Feedback (for

18. Requirement: OS X

19. Requirement: OS X
• macminicolo.net
• hosted-ci.com
• cisimple.com
• macincloud.com
• hostmyapple.com
• macminivault.com
• xcloud.me

20. Requirement: OS X
• macminicolo.net
• hosted-ci.com
t ni ght,
• cisimple.com un a
to r ac,
dul e CI n M
• macincloud.com
Or sche your ow ep...
on u sle
e yo
• hostmyapple.com whil
• macminivault.com
• xcloud.me

21. Security
sudo /usr/sbin/DevToolsSecurity --enable

22. Security
sudo xcodebuild -license

23. Credentials
• The CI server needs access to your
repository
• SSH key
• The CI server needs Keychain credentials if
you want to sign your builds
• Developer certificate and profiles

24. Keychain
• Export and Import using Keychain Access app
• Keys and Certificates
• Or use the command line:
• sudo security import /path/to/distribution.cer -k /Library/Keychains/
System.keychain
• sudo security import /path/to/distribution.p12 -k /Library/Keychains/
System.keychain
• Don’t forget about provisioning profiles
• Troubleshooting: http://tinyurl.com/iOSDevToolTroubles

25. Tip #2

26. Avoid this error
from Xcode
RunPlatformUnitTests:
warning: Skipping tests;
the iPhoneSimulator platform does
not currently support application-
hosted tests (TEST_HOST set).

27. The culprit
/Applications/Xcode.app/
Contents/Developer/
Platforms/
iPhoneSimulator.platform/
Developer/Tools/
RunPlatformUnitTests

28. The workaround(s)

29. Xcode Plugin
• https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin
• Specify provisioning profile
• Keychain access
• Signed IPA

30. Xcode Plugin
• Recommended xcodebuild arguments:
• GCC_SYMBOLS_PRIVATE_EXTERN=NO
• COPY_PHASE_STRIP=NO
• Allows test bundle to link with Release
build symbols

31. Signing with the Plugin

32. Signing with the Plugin

33. Signing with the Plugin

34. Tip #3

35. Prefer Scripts
over CI Plugins
• More flexibility
• More resilient to Apple’s changes
• More power to adapt and expand (specificity)
• Less coupling to the specific CI server type
• You can version control the settings easier

36. xcodebuild
xcodebuild
clean build

37. Tip #4

38. Be specific with
xcodebuild options

39. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

40. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

41. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

42. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

43. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

44. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

45. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

46. xcodebuild
xcodebuild
-scheme SOLID
-target SOLID
-configuration Debug
-arch i386
-sdk iphonesimulator
clean build

47. Tip #5

48. Specify Alternate
Output Locations
• Don’t rely on Xcode’s cryptic locations
• This will make your scripts easier to
manage

49. # Build location for Instruments test runs.
CI_BUILD_DIR=/tmp/SOLID_AUTOMATION
mkdir -p $CI_BUILD_DIR
xcodebuild
...
CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR
clean build

50. # Build location for Instruments test runs.
CI_BUILD_DIR=/tmp/SOLID_AUTOMATION
mkdir -p $CI_BUILD_DIR
xcodebuild
...
CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR
clean build

51. # Build location for CI test runs.
CI_BUILD_DIR=/tmp/SOLID_AUTOMATION
mkdir -p $CI_BUILD_DIR
xcodebuild
...
CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR
clean build

52. Demo

53. Tip #6

54. Ensure Return Codes
• If the tool you are invoking in your script
doesn’t return an exit code indicating
pass/fail, it’s up to you.
• Unix-friendly 0 (success) or 1 (failure)

55. Tip #7

56. Test against multiple
SDKs
-sdk iphonesimulator5.0
-sdk iphonesimulator5.1
-sdk iphonesimulator6.0

58. Other Tools
to Know About
• xcode-select (manage xcode path)
• Overridden by DEVELOPER_DIR
• xcrun (find and run dev tools)
• ibtool (for Interface Builder files)
• /Applications/Xcode.app/Contents/Developer/usr/bin/
• Variables in Xcode
• “Build Settings Reference”

60. OCUnit

61. Other Related Tools

62. Other Related Tools
• OCMock • XcodeTest
• OCMockito • AutoBuild
• OCHamcrest • xcodeproj (CocoaPods)
• Expecta • Circle?
• TinyMock • Simon?
• LRMocky • ios-maven-plugin
• OCUnit2JUnit • Ceedling
• OCUnitReport • XcodeCoverage
• Kicker • JMRTestTools
• xcodebuild-rb • Nocilla
• xcodearchive • OHHTTPStubs

63. OCUnit Alternatives
• Kiwi
• Cedar
• OCDSpec
• C++ Automated Test Cases in Headers
(CATCH)
• Objective-Shoulda
• Specta

64. Tip #8

65. OCUnit2JUnit for
Test Results
• ocunit2junit.rb
• https://github.com/ciryon/OCUnit2JUnit
• Converts output to JUnit format for easier
transformation to HTML

67. UIAutomation

68. Other UIAutomation
Tools
• tunup_js • ios-sim
• jasmine-iphone • ios-sim-locale
• uiautomation-jasmine- • iphonesim
iphone
• ui-auto-monkey
• Bwoken
• ui-screen-shooter
• Zucchini Framework
• uiautomation-rb?
• WaxSim

69. UIAutomation
Alternatives
• Frank • Objective C Slim
(ocslim)
• (iCuke)
• AutomationKit
• KIF
• Sikuli
• Calabash
• MonkeyTalk (formerly
• UISpec FoneMonkey)
• Bromine • Plus other commercial
tools, such as Telerik Test
Studio for iOS

71. Tip #9

72. PListBuddy
is Your Buddy
/usr/libexec/PlistBuddy

74. Tip #10

75. Always quit the
simulator
• killall
• killall -m -KILL "iPhone Simulator"
• AppleScript
• osascript -e 'tell app "iPhone Simulator"
to quit'

76. Tip #11

77. Use Jonathan Penn’s
AutomationExample
https://github.com/jonathanpenn/AutomationExample

78. Demo

79. More?

80. Code Coverage
“Risk coverage”

81. Coverage Tools
• gcov
• GCC’s coverage tool
• GUIs for gcov
• LCOV (HTML)
• CoverStory (Mac app)
• ocov?

82. Tip #12

83. gcovr for
Coverage Automation
Produces Cobertura-formatted output
from gcov data

84. Tip #12.1

85. Enable Test Coverage

87. Static Analysis
• Clang
• http://clang-analyzer.llvm.org/
• scan-build

88. Tip #13

89. Symbolic links
can be your ally

90. Avoid scan-build Pain
sudo ln -s
/Applications/Xcode.app/Contents/Developer/Toolchains/
XcodeDefault.xctoolchain/usr/lib/arc/
libarclite_iphonesimulator.a
/usr/lib/arc/libarclite_iphonesimulator.a

93. What about
test build deployment?

94. Tip #14

95. Use TestFlight
• testflightapp.com
• OTA deployment
• SDK for more features
• Free!!

96. TestFlight
• Web UI
• Desktop App
• REST API
• Jenkins Plugin

97. TestFlight Alternatives
• HockeyApp
• http://hockeyapp.net
• Also has a Jenkins plugin (forked)
• BetaBuilder for iOS
• http://www.hanchorllc.com/betabuilder-for-ios/
• More players in this space, esp. for enterprises...

98. TestFlight Config
• API Token and Team Token
• Need a signed IPA
• They get an email with a link for OTA
installation!

99. Tip #15

100. Protect Your Tokens
• API Token & Team Token
• Keep them out of scripts
• Instead, define them in Jenkins (and leverage
Jenkins security)
• 'Configure System'
• 'Global properties' section
• Or use the plugin (same parameters)

101. TestFlight Plugin

102. TestFlight Script

103. Deployment

104. Deployment

105. Q: Once you have a nightly job to build,
unit test, sign an IPA and deploy it to QA,
what’s next?

106. Q: Once you have a nightly job to build,
unit test, sign an IPA and deploy it to QA,
what’s next?
A: Sleeping easy.

107. Documentation
• Appledoc
• Doxygen
• HeaderDoc

108. DocSets

109. Appledoc

110. Doxygen

111. Tip #16

112. Don’t stop there!

113. Other Goodies

114. Other Goodies
• Ensure your site or API is up
• Scan your code for TODOs and FIXMEs
• Get trend reports for lines of code
(SLOCCount w/ sloc2html)
• Scan for duplicated blocks of code (CPD,
Simian, etc.)
• Use agvtool (Apple-Generic Versioning
Tool) to increment your build number

117. Still Missing
• Cyclomatic Complexity
• Coding convention/Style checker
• Code Formatter
• Uncrustify, UniversalIndentGUI
• UML Generation
• via CLI, as opposed to OmniGraffle (AppleScript?)
• ER Diagram from Core Data schema

118. Tip #n

122. Sources
• Slides: http://www.slideshare.net/muncman
• http://blog.carbonfive.com/category/mobile/
• http://www.stewgleadow.com/
• http://lifeandcode.net/
• http://longweekendmobile.com/blog/
• http://longweekendmobile.com/2011/04/17/xcode4-running-application-
tests-from-the-command-line-in-ios/
• http://qualitycoding.org
• et cetera

123. Thank you!
• CodeMash organizers
• CodeMash sponsors
• and YOU!
[MU]

124. Thank you!
• CodeMash organizers
• CodeMash sponsors
• and YOU!
Questions?
[MU]