Learn how to practice configuration as code by using the Job DSL plugin for Jenkins. Find out how to organize Job DSL scripts and apply code reuse and refactoring to your Jenkins configuration. This talk will cover advanced techniques for large scale installations and show how to extend the Job DSL for your favorite plugins.

1. Configuration as Code: The Job DSL Plugin
Daniel Spilker – CoreMedia AG

2. #jenkinsconf
Daniel Spilker
• Works for CoreMedia in Hamburg, Germany
• Software Architect on the Engineering Tools Team
• Maintainer of the
– Job DSL Plugin
– Gradle JPI Plugin

3. #jenkinsconf
Agenda
• Configuration as Code
• The Job DSL Plugin
• Q & A

4. #jenkinsconf
Current Situation
• No single job that builds everything
• Each branch needs its own pipeline
• Every team has their own jobs

5. #jenkinsconf
Problem
• Lots of copy&paste
• Editing in HTML
text areas
• Settings hidden behind
Advanced button
• Working with the UI
can be slow

6. #jenkinsconf
Configuration As Code
• Create new
pipelines quickly
• Refactor jobs
• Trace changes
• Work with your
favorite tool set

7. #jenkinsconf
There Is A Plugin For That
• Template Project Plugin
• Job Generator Plugin
• Literate Plugin
• JobConfigHistory Plugin
• Workflow Plugin
• Job DSL Plugin
• …
Open Icon Library / CC BY 3.0

8. #jenkinsconf
Job DSL Language

9. #jenkinsconf
Job DSL Language
job('job-dsl-plugin') {
scm {
github('jenkinsci/job-dsl-plugin')
}
steps {
gradle('clean build')
}
publishers {
archiveArtifacts('**/job-dsl.hpi')
}
}

10. #jenkinsconf
Job DSL Language
job('job-dsl-plugin') {
scm {
github('jenkinsci/job-dsl-plugin')
}
steps {
gradle('clean build')
}
publishers {
archiveArtifacts('**/job-dsl.hpi')
}
}

11. #jenkinsconf
Job DSL Language
job('job-dsl-plugin') {
scm {
github('jenkinsci/job-dsl-plugin')
}
steps {
gradle('clean build')
}
publishers {
archiveArtifacts('**/job-dsl.hpi')
}
}

12. #jenkinsconf
Job DSL Language
job('job-dsl-plugin') {
scm {
github('jenkinsci/job-dsl-plugin')
}
steps {
gradle('clean build')
}
publishers {
archiveArtifacts('**/job-dsl.hpi')
}
}

13. #jenkinsconf
Job DSL Plugin

14. #jenkinsconf
Job DSL Plugin
• Install Job DSL Plugin
• Create free-style project
• Add “Source Code Management”
• Add “Process Job DSLs” build step
• Configure scripts
• Run job

15. #jenkinsconf
Job DSL Plugin
• Install Job DSL Plugin
• Create free-style project
• Add “Source Code Management”
• Add “Process Job DSLs” build step
• Configure scripts
• Run job

16. #jenkinsconf
Job DSL Plugin
• Install Job DSL Plugin
• Create free-style project
• Add “Source Code Management”
• Add “Process Job DSLs” build step
• Configure scripts
• Run job

17. #jenkinsconf
Job DSL Plugin
• Install Job DSL Plugin
• Create free-style project
• Add “Source Code Management”
• Add “Process Job DSLs” build step
• Configure scripts
• Run job

18. #jenkinsconf
Job DSL Plugin
• Install Job DSL Plugin
• Create free-style project
• Add “Source Code Management”
• Add “Process Job DSLs” build step
• Configure scripts
• Run job

19. #jenkinsconf
Job DSL Plugin
• Install Job DSL Plugin
• Create free-style project
• Add “Source Code Management”
• Add “Process Job DSLs” build step
• Configure scripts
• Run job

20. #jenkinsconf
Batteries Included
125 plugins supported by the DSL

21. #jenkinsconf
Batteries Included
EnvInject
Groovy
Copy Artifact
Git
Subversion
Folders
Extra Columns
StashNotifier
Gradle
Build Pipeline
Workspace Cleanup
GitHub Pull Request Builder
GitHub
JaCoCoRelease
Build Flow
Robot Framework
Tool Environment
Conditional BuildStep
Throttle Concurrent Builds
Associated Files
Workflow
Emma
Xvnc
AnsiColor
Timestamper
Text-Finder
Job DSL Perforce
Ant
Maven Project

22. #jenkinsconf
Batteries Included
Checkstyle
Categorized Jobs View
Build-timeout
Config File Provider
Golang
Delivery Pipeline
HipChat
Naginator
Jabber
NestedViews
HTML Publisher
NodeJS
Flowdock
Credentials Binding
Plot
RunDeck
Rake
Powershell
SBT
xUnit
xvfb
Warnings
SonarvSphere Cloud
RVM
Port Allocator
Sectioned View
SSH Agent
Build Monitor
Claim
Email-ext
Findbugs
Grails
Javadoc
Multijob

23. #jenkinsconf
Community Driven
70 contributors 500 pull requests

24. #jenkinsconf
Extending The DSL

25. #jenkinsconf
Extending The DSL

26. #jenkinsconf
Extending The DSL

27. #jenkinsconf
Extending The DSL
job('example') {
...
configure { project ->
project / buildWrappers <<
EnvInjectPasswordWrapper {
injectGlobalPasswords(true)
}
}
}

28. #jenkinsconf
Job DSL Playground
• http://job-dsl.herokuapp.com/

29. #jenkinsconf
Everything is Groovy

30. #jenkinsconf
Everything is Groovy
def branches = ['master', 'feature-a']
branches.each { branch ->
job("jenkins-${branch}") {
scm {
github('jenkinsci/jenkins', branch)
}
…
}
}

31. #jenkinsconf
Using Libraries
• Any Java / Groovy library (JARs) can be used
• Download and dependency resolution must be
handled before running the Job DSL build step
• Anything can be used to download the JARs
– Build tools like Gradle
– Repository Connector Plugin
– Shell script with curl or wget

32. #jenkinsconf
Using Libraries
import org.kohsuke.github.GitHub
def gitHub = GitHub.connect()
def repo = gitHub.getRepository('jenkinsci/jenkins')
repo.branches.keys.each { branch ->
job("jenkins-${branch}") {
…
}
}

33. #jenkinsconf
Using Libraries – Gradle Example
repositories { jcenter() }
configurations { libs }
dependencies {
libs 'org.kohsuke:github-api:1.50'
}
task libs(type: Copy) {
into 'libs'
from configurations.libs
}

34. #jenkinsconf
Using Libraries – Gradle Example

35. #jenkinsconf
Using Functions
def createMavenJob(def jobFactory, def name) {
jobFactory.mavenJob(name) {
goals('clean verify')
jdk('Java 7 latest')
mavenInstallation('Maven 3.2.5')
publishers {
sonar()
}
}
}

36. #jenkinsconf
Using Functions
def createMavenJob(def jobFactory, def name) { … }
def jobA = createMavenJob(this, 'project-a')
def jobB = createMavenJob(this, 'project-b')

37. #jenkinsconf
Using Functions
def createMavenJob(def jobFactory, def name) { … }
def jobA = createMavenJob(this, 'project-a')
def jobB = createMavenJob(this, 'project-b')
jobB.with {
scm {
github('example-corp/project-a')
}
}

38. #jenkinsconf
Logging
println 'awesome!'
(1..10).each {
println "loop ${it}"
}

39. #jenkinsconf
Logging
class Test {
static demo(def out) {
out.println 'Must use out here!'
}
}
Test.demo(out)

40. #jenkinsconf
IDE Support

41. #jenkinsconf
IDE Support
• IntelliJ IDEA only
• Use a Gradle build file for configuration
apply plugin: 'groovy'
sourceSets { … }
repositories { … }
dependencies {
compile 'org.jenkins-ci.plugins:job-dsl-core:1.34'
}

42. #jenkinsconf
Syntax Highlighting

43. #jenkinsconf
Parameter Information

44. #jenkinsconf
Documentation

45. #jenkinsconf
Using Credentials

46. #jenkinsconf
Do not put
plain text credentials
in a DSL script

47. #jenkinsconf
Credentials Plugin
• Use the Credentials Plugin
for managing credentials
• The essentials plugins can
consume these credentials
(Git, Subversion, …)
• Use the Credentials Binding
Plugin to map credentials to
environment variables

48. #jenkinsconf
Using Credentials
job('example') {
scm {
git {
remote {
github('example-corp/example')
credentials('example-corp-github')
}
}
}
}

49. #jenkinsconf
The DSL In Depth

50. #jenkinsconf
Supported Project Types
// https://wiki.jenkins-ci.org/display/JENKINS/Maven+Project+Plugin
mavenJob(…) { … }
// https://wiki.jenkins-ci.org/display/JENKINS/Matrix+Project+Plugin
matrixJob(…) { … }
// https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin
multiJob(…) { … }
// https://wiki.jenkins-ci.org/display/JENKINS/Workflow+Plugin
workflowJob(…) { … }
// https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin
buildFlowJob(…) { … }

51. #jenkinsconf
Workflow Job Example
workflowJob('acme') {
definition {
cps {
script(readFileFromWorkspace('acme.groovy'))
sandbox()
}
}
}

52. #jenkinsconf
Reading Files
workflowJob('acme') {
definition {
cps {
script(readFileFromWorkspace('acme.groovy'))
sandbox()
}
}
}

53. #jenkinsconf
Extending The DSL – Project Types

54. #jenkinsconf
Extending The DSL – Project Types
job('multi-branch') {
configure { project ->
project.name = 'freestyle-multi-branch-project'
…
}
}

55. #jenkinsconf
Creating Views
listView('project-a') {
jobs {
regex('project-a-.+')
}
columns {
status()
weather()
name()
}
}

56. #jenkinsconf
Supported View Types
sectionedView(…) { … }
nestedView(…) { … }
deliveryPipelineView(…) { … }
buildPipelineView(…) { … }
buildMonitorView(…) { … }
categorizedJobsView(…) { … }

57. #jenkinsconf
Using Folders
folder('team-a')
job('team-a/compile') {
...
}
https://wiki.jenkins-ci.org/display/JENKINS/CloudBees+Folders+Plugin

58. #jenkinsconf
Best Practices

59. #jenkinsconf
Best Practices
• Start by converting a few jobs
• Create new jobs from DSL scripts
• Gradually convert all jobs to DSL scripts

60. #jenkinsconf
Best Practices
• Commit your DSL scripts to SCM
• Do not put plain text credentials in DSL scripts
• Use Groovy code to avoid repetition

61. #jenkinsconf
Further Information
• Documentation
https://github.com/jenkinsci/job-dsl-plugin/wiki
• Mailing List
https://groups.google.com/forum/?fromgroups#!forum/job-dsl-plugin
• Examples
https://github.com/sheehan/job-dsl-gradle-example
• Playground
http://job-dsl.herokuapp.com/

62. #jenkinsconf
Questions?

63. #jenkinsconf
Thank You
Daniel Spilker
daniel.spilker@coremedia.com
@daspilker
We‘re hiring
www.coremedia.com
@CoreMediaMinds