Who *is* Jenkins?
•
5 likes•1,853 views
A talk given to the San Francisco Jenkins Area Meetup (JAM) in January of 2016 on the current state of the Jenkins project and some ideas we're looking at for the future.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Who *is* Jenkins?
Similar to Who *is* Jenkins? (20)
Recently uploaded
Recently uploaded (20)
Who *is* Jenkins?
- 1. sd ©2015CloudBees,Inc.AllRightsReserved @jenkins_spb ©2016CloudBees,Inc.AllRightsReserved Who is Jenkins? Current state, common usage issues and where we’re going next Andrew Bayer CloudBees, Inc. San Francisco Jenkins Area Meetup January 2016 @jenkinsci January 20, 2016
- 2. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci > Whoami – Andrew Bayer • Background: Builds, tools, you name it. For years. A few places. ! • Jenkins OSS contributor – Ex-member of governance board, core contributor – Plugin dev and maintainer – Jenkins infrastructure work too ! • Now at CloudBees – Community, evangelism, development… @abayerTwitter: GitHub: https://github.com/abayer
- 3. sd ©2015CloudBees,Inc.AllRightsReserved @jenkins_spb ©2016CloudBees,Inc.AllRightsReserved Outline 3 • Introduction. What is Jenkins? • Common Jenkins usage issues • Why does *that* happen? How to overcome those issues? • Jenkins trends (community changes, Jenkins 2.0)
- 4. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Disclaimer ! • This presentation is not training material ! • The talk represents the speaker’s personal opinion ! • This opinion may differ from the official position of CloudBees or Jenkins community ! • I may or may not have any clue what I’m talking about. 4
- 5. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins 101 5 • 2005 – Hudson (Sun) • 2008 – It becomes popular, Duke's Choice Award • 2011 – Conflict with Oracle, renaming to Jenkins • You may have heard about that… =) ! • License: MIT (core) • https://github.com/jenkinsci/
- 6. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins is popular 6 http://stats.jenkins-ci.org/jenkins-stats/ • 100k+ installations • ~300 active committers (6 months) • ~3 new plugins every week • ~30 plugin releases every week
- 7. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Who is Jenkins? 7 Jenkins – Continuous Integration System Automation Framework
- 9. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Why “framework”? 9 System Flexible and Extensible Minimal OOTB functionality Solves basic use-cases only Requires customization Jenkins on its own is…
- 11. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci How does Jenkins grow? After setup 11
- 12. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci How does Jenkins grow? Small instance 12 Dozens of jobs Several nodes Small number of plugins Several users
- 13. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci How does Jenkins grow? It becomes popular 13 Hundreds of jobs Dozens of nodes Many plugins Dozens of users Complex configurations
- 14. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci 14By NASA, ESA, J. Hester and A. Loll (Arizona State University) (HubbleSite: gallery, release.) [Public domain], via Wikimedia Commons. https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/512px- Crab_Nebula.jpg” There’s Jenkins! Exploding! At some point… Other services have been affected as well…
- 15. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Why does Jenkins blow up? •Jenkins is scalable, but… •Default Jenkins setups are not scalable •Scalability requires proper system design 15
- 16. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Rule for admins #0. Design automation systems as internal products 16But there are issues in Jenkins as well…
- 17. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci User experience issues 17 Where are the docs? How to find a plugin? Is this plugin alive? How to configure THIS? Why do they ignore my issues? Where is the maintainer? Why is the UI so complicated? Why does the UI look like 2005?
- 18. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins Learning Curve 18 Basic scripts, CRON with UI Automation flows based on plugins Advanced features, Custom Plugins
- 19. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins weaknesses. Limited scalability 19 Node 1 Node 2 Node N . . . • Master handles the most of the build logic • Jobs on nodes ALWAYS put load on the master • Input/Output on master is also a bottleneck Multi- tenancy ??? Gearman Plugin (OSS) CloudBees Jenkins Platform
- 20. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins weaknesses. Growth Problems 20 Jenkins used to grow faster than its Dev community
- 21. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins weaknesses. Compatibility costs 21 • More than 5 years of backward compatibility • Hudson 2 plugins still work on Jenkins! ! • BUT: • Architecture design issues, complex APIs • Outdated libs • Outdated UI • Plugin integration issues
- 22. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci The problems can be partially solved! 22
- 23. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Rule for admins #1. Do not overload Masters! 23
- 24. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Rule #1. Do not overload Masters! • Use labels to specify target nodes ! ! ! ! ! • or Job Restrictions Plugin to secure the master 24
- 25. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Rule for admins #2. Test Jenkins changes before going live! 25
- 26. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Test servers •Must-have for large-scale instances •Target: best possible coverage 26 New feature/ plugin Bugfix Test server Production Showstoppers only Production mirror?
- 27. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Rule for admins #3. Use LTS versions on production 27
- 28. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins LTS releases • LTS are much more stable • Bugs backporting, 3 monthly patch releases 28
- 29. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci LTS Specifics •LTS releases still require testing – Potential regressions across major versions – SECURITY fixes cause “regressions” ! •3 months is not enough, what to do? – Custom-fork of Jenkins core [and plugins] – Enterprise-solutions based on Jenkins – Help community to increase coverage and to extend the support 29
- 30. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Rule for admins #4 30 Everything dies.
- 31. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Your Jenkins instance will die •Major risk: System and Infra failures •Updates may corrupt the system as well ⇒ALWAYS perform backups – Periodic Backup Plugin, Backup Plugin, … ! •Config History Plugin – configs backup •Naginator Plugin – failover of jobs 31
- 32. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Job Config History Plugin – configuration backup 32 •Jenkins System Configs •Jobs •Nodes
- 33. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Monitoring •Monitoring Plugin (Java Melody) •Reporting to external systems –Nagios, Zenoss, Zabbix, … •Centralized log collection & processing 33 Don’t forget about the infrastructure and backend - they frequently fail !
- 34. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Summary • Jenkins has real weaknesses, it’s true. But… • Jenkins is effective if you know how to solve them ! • Useful materials: – http://jenkins-ci.org - Blogs, Wiki, Bugtracker – Materials from Jenkins User Conferences: o https://www.cloudbees.com/company/events/juc – Google Search – Need help? o IRC: #jenkinsci o https://groups.google.com/forum/#!forum/jenkinsci-users 34
- 35. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Automation Systems Evolvement Trends •SW area shifts focus from CI: – Continuous Delivery – DevOps ! •Many integrations => communities and OSS ! •Centralization of automation systems – High requirements and strict SLAs – Commercial solutions 35
- 36. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Community reorganization ! •New teams focusing on hot areas ! •Community manager ! •More contributors (mostly plugin developers) ! •Jenkins Board re-elections 36 Board Core Team Security LTS Community INFRA Website
- 37. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci “Adopt a plugin!” 37 http://bit.ly/adoptplugin
- 38. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Automation as Code 38 Job DSL DotCI* Pipeline Jenkinsfile + Multi-branch *not available in the main Update Center
- 40. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Plans under discussion (not a final roadmap) 40 Jenkins 1.x “Jenkins 3.x” Jenkins 2.x Middle of 2016 End of 2016 (?) • Website • Focus on UX and project infra • Minimal compatibility changes • Major architectural changes • At least 1 LTS line in 2016
- 41. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins 2.0. Website ! •New domain name! http://jenkins.io (in the works!) ! •Static web-site with sources on GitHub ⇒Improves contributions ! •Content rework – MORE structured info – Going beyond Continuous Integration 41 http://bit.ly/jenkins20
- 42. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Jenkins 2.0. What’s in the core? •UX: – “Plugin gallery” – Web UI improvements (configs, etc.) ! •Update of WebUI libraries •API deprecation and deletion engine 42 http://bit.ly/jenkins20
- 44. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci «Jenkins 3.0» •Aim– architecture changes ! •Features list– TBD – Database storage for configs and build data (Andrew Bayer) – Remoting 3.0 (Oleg Nenashev) – … – <feel free to propose changes> 44
- 45. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci How to contribute? 45 http://bit.ly/jenkinscidev1. GOTO Jenkins mailing list: 2. Search for Jenkins 2.0 tracks:
- 47. sd ©2016CloudBees,Inc.AllRightsReserved @jenkinsci Useful links • Jenkins project – Website: http://jenkins-ci.org – GitHub: http://github.com/jenkinsci/ ! • CloudBees – Website: https://www.cloudbees.com – CloudBees Jenkins Platform o https://www.cloudbees.com/products/cloudbees-jenkins-platform 47