How we made jenkins community

©2010 CloudBees, Inc. All Rights
Reserved
Reserved
How we made the Jenkins community
Kohsuke Kawaguchi
Creator of Jenkins / Architect, CloudBees, Inc.

Reserved
2©2010 CloudBees, Inc. All Rights
Reserved
What is Jenkins?
• Java web application
• For builds, tests, and broader automation
– Called “Continuous Integration server”
• MIT-licensed, 8 years old
jenkins-ci.org

Reserved
Reserved
Where we are today
• 53,000+ installations around the world
• 600+ plugins by 500 contributors
• #1 CI server in the market

Reserved
Reserved
How did we come this far?
• All the usual things
– Software that works
– Being at the right place at the right time
• But I‟d like to think there‟s more to it
– And it‟s something I think others can
replicate

Reserved
Reserved
Communication is inherently painful
• Ask if the project likes the change I‟m
thinking
• Work on the change
• Submit the patch and ask for approval
• Defend the patch and revise it
• Docs, tests, IP assignment
• Done

Reserved
Reserved
Programmers are picky bunch
• All sorts of things to quarrel about
– Should „{‟ be on the same line as “if”?
– Different taste for design trade-off
– Different anticipation for
future expansions
• Hardly bearable even
when you know them
personally

Reserved
Reserved
More so in FLOSS
• We don‟t know each other
• No relationship outside coding
• No common goals
• No command structure
• Time zone / language barrier

Reserved
Reserved
Communication is root of all evil
• Less, the better
• Get rid of the need for it
– Don‟t stand in the way of people and their
itches
– Innovations happen everywhere. Don‟t kill
them before it materializes

Reserved
Reserved
The Question

Reserved
Reserved
It starts with modularity
• It means:
– Large software, divided into small silos
– Module interaction controlled by contracts
– Code/build/test at module level
• Modularity enables extensibility

Reserved
Reserved
Extensibility!
• Open up to the rest of the world
• The only thing that really matters
– It enables all kinds of social/technical tricks
• It‟s not free, but it‟s worth the effort

Reserved
Reserved
#0: Programmability / Scriptability
• Every software should be a library
– Let other programs talk to your program
– There‟s always a layer above you
• “App” people tend to forget this
– ImageMagick vs Gimp

Reserved
Reserved
#1: Code loading at runtime
• Define extension points
– Contracts to be followed by extensions
• Discovering implementations
– Java, etc. requires some work
• Provide isolation

Reserved
Reserved
#2: Treat extensions as 1st class citizen
• If core can do something that extensions
can‟t, that‟s a bug
• If users can tell apart extensions from
core, you aren‟t doing enough

Reserved
Reserved
#3: Data model extensibility
• Extensions need to be able to store stuff
– By adding stuff to core domain objects
– By adding more domain objects
– For Jenkins, XML really helped
• Schema versioning becomes non-linear

Reserved
Reserved
#4: Embrace heterogeneousness
• Good contract enables multiple impls
– Take something, slightly change something,
and you have a new extension impl
– You see a list, I see extensibility
• And inheritance
• This is surprisingly tricky
in some context
– In webapp, for example

Reserved
Reserved
#5: Turtles all the way down
• Extensibility needs to be recursive
– Extensions should define its own extension
points
• In Jenkins, we provide an idiom
• Partly culture
• Partly refactoring

Reserved
Reserved
Symptom of fake modularity/extensibility
• Division of labor is not modularity
• Modules are tailor-fit to their neighbors
• Contracts aren‟t general enough
• A fixed set of modules become the
product
Mosaic I - M.C. Escher

Reserved
Reserved
Extensibility in Jenkins
• Core
– Creates structure
– Includes Modules (plugins in disguise)
• 600+ Plugins
– Actually provide useful CI functionalities

Reserved
Reserved
The Question

Reserved
Reserved
http://www.flickr.com/photos/jvuokko/4566270758/

Reserved
Reserved
Extensible silo is a good thing
• It forces contracts at boundaries
– Principles of “continuous X”: do it often enough
and it becomes painless
• Small code is easier to hack
– Looking at 5 files feel easier than 500 files
– Faster to build, easier to maintain
• Special use case does not clutter general-
purpose code/UI
– Example: Exclusion in Jenkins Subversion
plugin
• Niche features are welcome

Reserved
Reserved
Extensible silo requires less communication
• Everyone gets his own sandbox to hack
– No need to discuss
designs/features/schedules upfront
• No more conflicting development model
– Conference-driven development vs test-
driven development
• No need to see someone else‟s ugly
code
– No need to feel ashamed of it

Reserved
Reserved
Extensible silo fosters innovations
• Every whacky idea gets a shot
– No need to sell ideas
– No need to turn down ideas
• Communicate in code, not in English
• Some of them will turn out great
– No need to drag failed experiments

Reserved
Reserved
Extensibility highlights higher-order
problems
• With enough silos, core devs become silo
devs
• What core does directly benefits silos
– Simplified build mechanism
– IDE support
– Error checker for common mistakes
• It accelerates dev scalability

Reserved
Reserved
Lower barrier to entry, enabled by silo
• In Jenkins, everyone gets to be a committer
– Just ask, and it‟ll be given
– We even have an IRC bot that does this
• This works, because…
– People aren‟t collaborating
– Any damage done is compartmentalized
– VCS enables rollback so long as it‟s not
released
• After all, unless you are Linus Torvalds, “too
many committers” is not your problem

Reserved
Reserved
Lower barrier to entry: observations
• People are cautious when they have
power
– Very few incidents
• Works well when people move on
– Abandoned plugins get new maintainers
• It encourages small contributions
• You can never predict who becomes
significant contributors

Reserved
Reserved
But silo alone doesn‟t create a
community
Image: NASA/JPL-Caltech

Reserved
Reserved
You need a center of gravity
• Something that:
– Drag everything
together
– Pulls in people
• Shared source tree
repo doesn‟t have to be
“it”
Image: NASA/JPL-University of Arizona

Reserved
Reserved
Center of Gravity in Jenkins: IRC
• Closest thing to the water cooler in OSS
– It is a place people can come, unlike Twitter
– You can be present & absent at the same
time
• It builds human relationships

Reserved
Reserved
Center of Gravity in Jenkins: outreach
• Connecting people
– Meet-ups, Hackathon, Events
• Giving the project a voice
– Twitter, Blog
Mad props to @agentdero

Reserved
Reserved
But above all: Update Center
• UI built into Jenkins to let users install
plugins
• To be listed, your plugins need to be co-
located in GitHub org
• Commit access covers all the repos
• Everyone can add others to the project

Reserved
Reserved
Observations
• Creates an incentive for devs to come
• When people move on,
– Keeping plugin alive is easier
• A structure helps new contributors
– Right thing is now the easiest thing
• Providing pin-point technical help is
easier

Reserved
Reserved
Center of Gravity = Self-reinforcing
• As we acquire more plugins…
– We attract more users
– UC becomes more attractive to devs
• Machinery gets oiled
– As a result of solving higher-order problem

Reserved
Reserved
Emerging Challenges
• Subversion → Git changed dynamics
– Repository no longer projects enough gravity
• GitHub and pull requests
– It‟s great that contributions are easier
– But it also creates “throw the stuff over the
wall” mentality
– It lets people contribute without joining,
reducing the inflow

Reserved
Reserved
Conclusions
• Scale development with extensibility
• Reduce communication with extensibility
• Let people innovate with extensibility
• I think I made Steve proud
Extensibility! Extensibility!

How we made jenkins community

Recommended

Recommended

More Related Content

What's hot

What's hot (9)

Viewers also liked

Viewers also liked (6)

Similar to How we made jenkins community

Similar to How we made jenkins community (20)

More from Kohsuke Kawaguchi

More from Kohsuke Kawaguchi (19)

Recently uploaded

Recently uploaded (20)

How we made jenkins community

Editor's Notes