Uber mobility - Build & Release
•Download as PPTX, PDF•
10 likes•4,403 views
Speakers: Chirayu Krishnappa & Daniel Liem (Uber), Christian Legnitto (Pocketship), Jacek Suliga (LinkedIn), Robbert Van Ginkel & Gautam Korlam (Uber), Rachel Brindle (Pivotal Labs) Tools for building and releasing mobile applications provided by Google and Apple are geared towards smaller mobile application development teams. However, once development teams start to grow, the requirement from your build and release tooling change drastically. This meetup was all about building & releasing mobile applications at scale: How does your build infrastructure need to evolve to support increasingly complex code-bases? How can you build an automated release pipeline that tracks every build and rolls out your mobile applications safely? What kind of unexpected problems can arise from adopting new technologies when you have a larger team?
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Uber mobility - Build & Release
Similar to Uber mobility - Build & Release (20)
Recently uploaded
Recently uploaded (6)
Uber mobility - Build & Release
- 1. Daniel Liem & Chirayu Krishnappa Ship Fast & Stable @ Uber Scale 11.1.2016
- 5. 99.99%
- 6. ● Have a dedicated Release team ● Aggressive weekly release cadence ● Build cuts (CI) from Master ● Nightly (alpha) vs. Beta / Production builds ● Internal beta dogfooding vs. External Beta testing ● FF (Feature Flagging) wherever possible ● Avoid alphafixes, betafixes, hotfixes & rollbacks ● Soft Upgrades vs. Force Upgrades Our Process
- 8. Staged Rollouts iOS Sanity Tests / Patches Android Sanity Tests / Patches
- 9. Staged Rollouts iOS Sanity Tests / Patches Upload to iTunes, enable Testflight (Weekend dogfooding) Android Sanity Tests / Patches β channel (Weekend dogfooding)
- 10. Staged Rollouts iOS Sanity Tests / Patches Upload to iTunes, enable Testflight (Weekend dogfooding) Wait for Approval Android Sanity Tests / Patches β channel (Weekend dogfooding) Staged Rollouts (1% → 10% → 50% → 100%)
- 11. Staged Rollouts iOS Sanity Tests / Patches Upload to iTunes, enable Testflight (Weekend dogfooding) Launch 100% (All users at once!) Wait for Approval Android Sanity Tests / Patches β channel (Weekend dogfooding) Staged Rollouts (1% → 10% → 50% → 100%) (Partner app only) Soft Upgrade + Force Upgrade
- 15. How many apps?
- 17. How long does it take to “deploy to production”?
- 18. Build, sign, and more. Mostly deterministic.
- 19. Submit to App Store for approval
- 20. WAIT … !
- 22. WAIT … ! Your users will upgrade…eventually.
- 24. SPEED Not the speed of your mobile app
- 26. SPEED Speed of reactions when you discover issues
- 29. How many versions of your app are out there?
- 33. Need for a trusted system
- 34. Goal: Develop at full speed!
- 35. High quality bar! Goal: Develop at full speed
- 36. High quality bar! Goal: Develop at full speed Because slow to deploy/rollback
- 37. Signals
- 38. Signals
- 39. Soon to be 100s of signals Signals
- 40. How can you stay on top? How do you react to these signals?
- 41. Ticketing system at the core If there’s a failure that should block the train, there’s a ticket for it.
- 43. Block specific versions Verify patched versions Track mitigation tasks Weekly reports
- 45. Intelligent Subsystems crash detection at alpha stages vs. production Every alpha crash gets a ticket
- 46. Feature Flagging Automate to track features per version and turn off based on classified crashes
- 47. Reminders E.g. When a new build is ready (also in-app “upgrade” notifications)
- 49. Notifications/Alerts We’re pushing at 4pm It’s 2pm and we have new / unresolved tickets.
- 51. Production alerts are separate e.g. Spike in crashes, E2E, App Store ratings dropped.
- 53. Stage Everything Rollout to employees Test flight, alpha, beta channels
- 54. Stage Everything Alpha channel rollout? Stage it. 1%, 10%, 50% 100%
- 55. Stage Everything In app upgrade prompt? Stage that too! 1%, 10%, 100%.
- 56. In a nutshell… Deploys are slow Collect all signals Ticketing at core Automated reminders Ad hoc notifications / alerts
- 57. Thank you Proprietary and confidential © 2016 Uber Technologies, Inc. All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber. Ship Fast & Stable @ Uber Scale Daniel Liem & Chirayu Krishnappa
- 58. Building At Uber Scale Like a ‘BAUS’ Robbert van Ginkel & Gautam Korlam 11.1.2016
- 59. Overview Challenges with mobile development at scale Team Size Build Time Infrastructure Improving Developer Experience while tackling scale Architecture Workflow
- 60. Mobile Scale @ Uber
- 61. Several Hundred Mobile Developers Hundreds of commits daily 50% of code changes every month Shared modular codebase with hundreds of modules
- 62. Team Size
- 63. Commits
- 64. Architecting for Scale Code architecture Features are built as Plugins and shared between apps Code infrastructure and tooling Monorepo helps with modularization and sharing Regressions block the whole team Always keep master green Guard as much as possible at compile time Fail fast
- 65. Workflow at Scale Asynchronous change merging Submit queue Stacked Diffs Run expensive code quality checks pre merge UI Tests Deep static analysis - Infer etc. Performance regressions on real devices - cold start, battery, network etc.
- 66. Build Time
- 68. With more modules, come more problems
- 69. Build Tools Scaling Issues Cocoapods Does not scale well with more targets (15 min pod install time) Xcode Incorrect incremental builds (non deterministic and hard to debug) Xcode project file merge conflicts Gradle Does not scale well with large android projects (15 min for a single line change) Android Studio performance degrades
- 70. Building at Scale Both iOS and Android use Buck to build at Uber Incremental everywhere Scale non exponentially as more code is added Cache immutable state - avoid rebuilding Transparent Dependency Management Works well for monorepo iOS - Clean ~4x faster, Incremental ~20x faster Android - Clean ~6x faster, Incremental ~30x faster Remote Build Cache
- 71. Infrastructure
- 72. Uber’s CI Infrastructure CI capacity needs increased exponentially 400+ Busy Executors on CI hourly 50k+ CI Jobs run per day 600 400 200 100 CI Executors with Time
- 73. Optimizing the CI Pipeline Perform relevant checks at the right stage Code Formatting - pre diff Build, Unit Tests - diff UI, Static Analysis - pre merge Use CI resources effectively Remote build artifact caching Build in elasticity to meet peak demand
- 74. Open Source
- 75. Projects to Watch OkBuck - Gradle plugin that lets you use gradle projects with buck Buck Http Cache - A distributed build artifact cache service Coming soon: Swift Support in Buck https://github.com/uber/buck-http-cache 1https://github.com/uber/okbuck , Slide Deck
- 76. Takeaway Invest in the right build tools early on Scaling hardware only works till a certain point Having shared workflow/tools across platforms helps a lot in the long run Fail earlier and keep master always green
- 77. Scaling the Build Process at Uber Robbert van Ginkel & Gautam Korlam Thank You
- 78. In-product features for release engineers Christian Legnitto christian@pocketship.com
- 85. In-product features for release engineers
- 86. In-product
- 88. Empathy Agency
- 89. Agency Fix Release Engineering pain Gain more code context and confidence Improve Release Engineering processes Influence company direction
- 90. Empathy Build times and CI turnaround Tests, test infrastructure, and tooling Release process overhead Shipping to the world
- 93. Specific features Company Product Release
- 94. Crash, OOM, hang reporting
- 96. Feature flags (A/B testing) if (whatever) { doA(); } else { doB(); }
- 97. Feature flags (A/B testing) if (whatever) { doA(); } else { doB(); }
- 98. Feature flags (A/B testing)
- 99. Bug reporter
- 100. Bug reporter
- 101. Bug reporter
- 102. Bug reporter
- 103. Bug reporter
- 104. Bug reporter
- 105. Promotion framework
- 106. Promotion framework
- 107. Promotion framework
- 108. Promotion framework
- 109. Promotion framework Test the next version of Facebook Get access to new features and bug fixes before everyone else.
- 110. Promotion framework
- 111. Promotion framework
- 112. Promotion framework
- 114. React Native
- 116. React Native JS Bundle JS Bundle ↑
- 117. React Native
- 118. WebViews
- 119. WebViews
- 125. Version number scheme Ship production weekly Slow rollout Alpha and beta Internal employee dogfooding
- 128. Version number scheme 100.0.0.20.70 Major version
- 129. Version number scheme 100.0.0.20.70Major version
- 130. Version number scheme 100.0.0.20.70Major version Hotfix
- 131. Version number scheme 100.0.0.20.70Major version Hotfix
- 132. Version number scheme 100.0.0.20.70Major version Hotfix Beta
- 133. Version number scheme 100.0.0.20.70Major version Hotfix Beta
- 134. Version number scheme 100.0.0.20.70Major version Hotfix Alpha Beta
- 135. Version number scheme 100.0.0.20.70Major version Hotfix Beta Alpha
- 141. THE DARK SIDE OF ENTERPRISE SWIFT Jacek Suliga 🍺 Mobilize @ LinkedIn
- 146. Builds Perf & App Size Launch Performance Maintenance Cost
- 147. Builds Perf & App Size Launch Performance Maintenance Cost
- 148. 8-23 MB 10-30%
- 151. 20% slower
- 153. Builds Perf & App Size Launch Performance Maintenance Cost
- 154. WWDC 2016, 406, “Optimizing app startup time”
- 157. Builds Perf & App Size Launch Performance Maintenance Cost
- 158. 0. 5 1. 0 1. 1 1. 2 2. 0 2. 1 Open Sourc e 2. 2 2. 3 3. 0 6/14 9/14 10/14 4/15 6/15 9/15 12/15 3/16 6/16 9/16
- 163. 3. 0
- 164. 0. 3
- 167. Any Questions?!
- 168. Automating Mobile Releases Rachel Brindle
- 169. TERMS • User: Someone who uses your app. • Environment: Place where a user can download your app • Staging: HockeyApp, Testflight, etc. • Production: Enterprise MDM, App Store, Play Store • Automated Deployment: Using CI to push to environment z
- 170. Processz
- 171. WHY • Frees up the deployer to do other things • Consistent deploys • Shorter release cycle • Documents $ git push pushed g4edeff4 $ # wait $ rake check_if_deployed Latest is g4edeff4 z
- 172. • Make sure tests pass • Build for release • Gather metadata • Screenshots • Release Notes • Other? • Upload to environment
- 173. • Make sure tests pass • Build for release • Gather metadata • Screenshots • Release Notes • Other? • Upload to environment