Submit Search
Upload
Jenkins Workflow Summit London 2015: Kohsuke/Jesse slides
•
10 likes
•
5,556 views
J
jgcloudbees
Follow
http://www.meetup.com/jenkinsmeetup/events/219967721/ slides from Kohsuke & Jesse
Read less
Read more
Technology
Report
Share
Report
Share
1 of 28
Download now
Download to read offline
Recommended
Jenkins days workshop pipelines - Eric Long
Jenkins days workshop pipelines - Eric Long
ericlongtx
Down the Rabbit Hole: An Adventure in JVM Wonderland
Down the Rabbit Hole: An Adventure in JVM Wonderland
Charles Nutter
Idiomatic gradle plugin writing
Idiomatic gradle plugin writing
Schalk Cronjé
XSLT and XPath - without the pain!
XSLT and XPath - without the pain!
Bertrand Delacretaz
Living with Garbage by Gregg Donovan at LuceneSolr Revolution 2013
Living with Garbage by Gregg Donovan at LuceneSolr Revolution 2013
Gregg Donovan
Super Charged Configuration As Code
Super Charged Configuration As Code
Alan Beale
Jenkins Job DSL plugin
Jenkins Job DSL plugin
Nikita Bugrovsky
Continuous Delivery of Puppet-Based Infrastructure - PuppetConf 2014
Continuous Delivery of Puppet-Based Infrastructure - PuppetConf 2014
Puppet
Recommended
Jenkins days workshop pipelines - Eric Long
Jenkins days workshop pipelines - Eric Long
ericlongtx
Down the Rabbit Hole: An Adventure in JVM Wonderland
Down the Rabbit Hole: An Adventure in JVM Wonderland
Charles Nutter
Idiomatic gradle plugin writing
Idiomatic gradle plugin writing
Schalk Cronjé
XSLT and XPath - without the pain!
XSLT and XPath - without the pain!
Bertrand Delacretaz
Living with Garbage by Gregg Donovan at LuceneSolr Revolution 2013
Living with Garbage by Gregg Donovan at LuceneSolr Revolution 2013
Gregg Donovan
Super Charged Configuration As Code
Super Charged Configuration As Code
Alan Beale
Jenkins Job DSL plugin
Jenkins Job DSL plugin
Nikita Bugrovsky
Continuous Delivery of Puppet-Based Infrastructure - PuppetConf 2014
Continuous Delivery of Puppet-Based Infrastructure - PuppetConf 2014
Puppet
Managing Jenkins with Jenkins (Jenkins User Conference Palo Alto, 2013)
Managing Jenkins with Jenkins (Jenkins User Conference Palo Alto, 2013)
Gareth Bowles
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Puppet
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Andrey Devyatkin
Los vatos
Los vatos
losvatoslocos
Juc west-how to build a jenkins db the wrong way!
Juc west-how to build a jenkins db the wrong way!
Michael Barbine
Jenkins Workflow - An Introduction
Jenkins Workflow - An Introduction
Ben Snape
Writing a Jenkins / Hudson plugin
Writing a Jenkins / Hudson plugin
Anthony Dahanne
20160929 android taipei Sonatype nexus on amazon ec2
20160929 android taipei Sonatype nexus on amazon ec2
TSE-JU LIN(Louis)
groovy and concurrency
groovy and concurrency
Paul King
Using Jenkins XML API
Using Jenkins XML API
Anton Weiss
DevOps – SonarQube
DevOps – SonarQube
Delta-N
Jenkins 101: Continuos Integration with Jenkins
Jenkins 101: Continuos Integration with Jenkins
All Things Open
Continuous Cross Platform Mobile App Development using Jenkins Build Server
Continuous Cross Platform Mobile App Development using Jenkins Build Server
Adam Paxton
Track code quality with SonarQube
Track code quality with SonarQube
Dmytro Patserkovskyi
Jenkins Scriptler in 90mins
Jenkins Scriptler in 90mins
Larry Cai
Groovy Maven Builds
Groovy Maven Builds
Evgeny Goldin
Continuous Development Pipeline
Continuous Development Pipeline
Izzet Mustafaiev
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
jgcloudbees
Maven Nexus
Maven Nexus
ericndunn
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
LoriGlavin3
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
LoriGlavin3
More Related Content
Viewers also liked
Managing Jenkins with Jenkins (Jenkins User Conference Palo Alto, 2013)
Managing Jenkins with Jenkins (Jenkins User Conference Palo Alto, 2013)
Gareth Bowles
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Puppet
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Andrey Devyatkin
Los vatos
Los vatos
losvatoslocos
Juc west-how to build a jenkins db the wrong way!
Juc west-how to build a jenkins db the wrong way!
Michael Barbine
Jenkins Workflow - An Introduction
Jenkins Workflow - An Introduction
Ben Snape
Writing a Jenkins / Hudson plugin
Writing a Jenkins / Hudson plugin
Anthony Dahanne
20160929 android taipei Sonatype nexus on amazon ec2
20160929 android taipei Sonatype nexus on amazon ec2
TSE-JU LIN(Louis)
groovy and concurrency
groovy and concurrency
Paul King
Using Jenkins XML API
Using Jenkins XML API
Anton Weiss
DevOps – SonarQube
DevOps – SonarQube
Delta-N
Jenkins 101: Continuos Integration with Jenkins
Jenkins 101: Continuos Integration with Jenkins
All Things Open
Continuous Cross Platform Mobile App Development using Jenkins Build Server
Continuous Cross Platform Mobile App Development using Jenkins Build Server
Adam Paxton
Track code quality with SonarQube
Track code quality with SonarQube
Dmytro Patserkovskyi
Jenkins Scriptler in 90mins
Jenkins Scriptler in 90mins
Larry Cai
Groovy Maven Builds
Groovy Maven Builds
Evgeny Goldin
Continuous Development Pipeline
Continuous Development Pipeline
Izzet Mustafaiev
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
jgcloudbees
Maven Nexus
Maven Nexus
ericndunn
Viewers also liked
(19)
Managing Jenkins with Jenkins (Jenkins User Conference Palo Alto, 2013)
Managing Jenkins with Jenkins (Jenkins User Conference Palo Alto, 2013)
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Los vatos
Los vatos
Juc west-how to build a jenkins db the wrong way!
Juc west-how to build a jenkins db the wrong way!
Jenkins Workflow - An Introduction
Jenkins Workflow - An Introduction
Writing a Jenkins / Hudson plugin
Writing a Jenkins / Hudson plugin
20160929 android taipei Sonatype nexus on amazon ec2
20160929 android taipei Sonatype nexus on amazon ec2
groovy and concurrency
groovy and concurrency
Using Jenkins XML API
Using Jenkins XML API
DevOps – SonarQube
DevOps – SonarQube
Jenkins 101: Continuos Integration with Jenkins
Jenkins 101: Continuos Integration with Jenkins
Continuous Cross Platform Mobile App Development using Jenkins Build Server
Continuous Cross Platform Mobile App Development using Jenkins Build Server
Track code quality with SonarQube
Track code quality with SonarQube
Jenkins Scriptler in 90mins
Jenkins Scriptler in 90mins
Groovy Maven Builds
Groovy Maven Builds
Continuous Development Pipeline
Continuous Development Pipeline
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
Maven Nexus
Maven Nexus
Recently uploaded
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
LoriGlavin3
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
LoriGlavin3
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
LoriGlavin3
How to write a Business Continuity Plan
How to write a Business Continuity Plan
Databarracks
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
Alfredo García Lavilla
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
Curtis Poe
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
Dubai Multi Commodity Centre
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
BookNet Canada
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Sergiu Bodiu
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Mark Simos
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
LoriGlavin3
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
Addepto
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
Fwdays
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
DianaGray10
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
Mattias Andersson
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
LoriGlavin3
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
Commit University
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Lonnie McRorey
Recently uploaded
(20)
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
How to write a Business Continuity Plan
How to write a Business Continuity Plan
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Jenkins Workflow Summit London 2015: Kohsuke/Jesse slides
1.
©2015 CloudBees, Inc.
All Rights Reserved©2015 CloudBees, Inc. All Rights Reserved Workflow in Jenkins Kohsuke Kawaguchi / CTO, CloudBees, Inc. kk@kohsuke.org / @kohsukekawa Jesse Glick / Developer, CloudBees, Inc. @tyvole
2.
©2015 CloudBees, Inc.
All Rights Reserved
3.
©2015 CloudBees, Inc.
All Rights Reserved Use Cases: orchestrated activities • Multi-stage continuous deployment pipeline • Run part of build with a temporary server • Blue/green deployment + auto commit/abort • Parallel tests with automatic sharding • Automatic per-branch jobs (à la Literate)
4.
©2015 CloudBees, Inc.
All Rights Reserved Characteristics • Complex pipelines involving multiple stages • Non-sequential logic such as loops and forks • Long-running builds must survive outages • Interaction with humans pauses, input • Restartable builds in case of a transient error • Reusable definitions to avoid duplication • Comprehensible scripts one clear definition
5.
©2015 CloudBees, Inc.
All Rights Reserved
6.
©2015 CloudBees, Inc.
All Rights Reserved
7.
©2015 CloudBees, Inc.
All Rights Reserved
8.
©2015 CloudBees, Inc.
All Rights Reserved node('linux') { stage 'Dev' git 'https://github.com/me/app.git' sh "${tool 'Maven 3'}/bin/mvn install" parallel regr: {sh './regression-test.sh'}, perf: {sh './performance-test.sh'} input 'All set?' stage name: 'Prod', concurrency: 1 sh './deploy.sh' }
9.
©2015 CloudBees, Inc.
All Rights Reserved Key features (what I already covered) • Entire flow is one concise Groovy script – for-loops, try-finally, fork-join, etc. • Can restart Jenkins while flow is running • Human input/approval integrated into flow
10.
©2015 CloudBees, Inc.
All Rights Reserved More key features • Allocate slave nodes and workspaces – as many as you like, when you like • Standard project concepts: SCM, artifacts, …
11.
©2015 CloudBees, Inc.
All Rights Reserved Design: workflow in VCS (1) • Load the script from another file • In repository node { git 'https://github.com/acme/foo.git' load 'jenkins.groovy' } pom.xml src jenkins.groovy ...
12.
©2015 CloudBees, Inc.
All Rights Reserved Design: workflow in VCS (2) • Load the whole script from repository • Job definition contains no Groovy
13.
©2015 CloudBees, Inc.
All Rights Reserved Design: workflow in VCS (3) • Shared workflow scripts/utilities in Git • Your job just needs to have this Git org.acme.MyWorkflow.run(this)
14.
©2015 CloudBees, Inc.
All Rights Reserved Stages (aka James Nord operator) • Pipeline throttling primitive • Special semaphore: only newest build may wait time builds build build build selenium tests selenium tests selenium tests build selenium tests deployment deployment build build selenium tests deployment all but deployment phase runs in parallel deployment still runs as fast as it can canceled
15.
©2015 CloudBees, Inc.
All Rights Reserved Build #153Build #153
16.
©2015 CloudBees, Inc.
All Rights Reserved foo() void foo() { try { sh 'sleep 1h' } finally { sh 'rm -rf *.tmp' } }
17.
©2015 CloudBees, Inc.
All Rights Reserved Resumption of Groovy flows • Transformed to “continuation-passing style” • Run on custom interpreter of Groovy – on master; includes security checks • State of program saved at each pause point • Variables serialized and restored after restart
18.
©2015 CloudBees, Inc.
All Rights Reserved Resumed builds to the user • It “just works” • (Serializable) local variables restored too • Shell-like steps survive restart – Reconnection of slave, too
19.
©2015 CloudBees, Inc.
All Rights Reserved
20.
©2015 CloudBees, Inc.
All Rights Reserved Jenkins Enterprise by CloudBees • CloudBees’ value-added Jenkins distro – Support & additional plugins • Resume from checkpoint – Can pick up artifacts from original build – No need to rerun earlier expensive steps • Visualization – “Stage” view of pipelines – More planned later • Templates – Define script in template, use attributes
21.
©2015 CloudBees, Inc.
All Rights Reserved Demo
22.
©2015 CloudBees, Inc.
All Rights Reserved Still to come • more plugin integrations • lots of little RFEs & bugs • easier script syntax, editing • workspace management • multibranch projects? • matrix project equivalent? • much more, tracked in JIRA
23.
©2015 CloudBees, Inc.
All Rights Reserved Status • Open for contribution • github.com/jenkinsci/workflow-plugin • 1.3 current release • Requires Jenkins 1.580.x+ today • Under active development
24.
©2015 CloudBees, Inc.
All Rights Reserved Summary • Simplifies complex orchestration • Define everything in single job • Workflow as Groovy code • Survives JVM loss, and even restartable • Extensible • github.com/jenkinsci/workflow-plugin
25.
©2015 CloudBees, Inc.
All Rights Reserved
26.
©2015 CloudBees, Inc.
All Rights Reserved Design: flows • persistent record of execution • directed acyclic graph of nodes • some nodes represent one step • others indicate block start/stop structure • nodes may have associated metadata – console log fragment contributes to main log • pluggable visualizations for different views
27.
©2015 CloudBees, Inc.
All Rights Reserved Design: steps • standalone API for asynchronous build steps • context serves as an identifier & callback – also offers logger, build, workspace, &c. • support for block-structured steps – invoke body 0+ times with revised context • standard step for “durable” shell/batch scripts • standard steps for SCMs (git, svn, other) – >1 SCM per build possible
28.
©2015 CloudBees, Inc.
All Rights Reserved Design: interoperability • run on existing Jenkins slaves • SCM plugins supported with modest changes • some existing build steps & publishers – “build wrappers” coming (1.599+) • trigger existing jobs • standard build history, artifacts • needs Jenkins 1.580.x or later – features factored out of standard projects
Download now