Taking Jenkins Pipeline to the Extreme

Taking Jenkins Pipeline to the Extreme
Yinon Avraham
Tech Lead @ eBay
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.com

AGENDA
● Why
● What
● How
● WOW!
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference

WHY

MICROSERVICES
Service
Service
Service Service
SQL
DB
Service
SQL
DB
Service
NoSQL
DB
HTTPHTTP
HTTP HTTP
3rd Party
Service

WHAT

IMPROVED CI/CD PROCESS
CI: For each active branch:
1. Build
2. Unit Tests
3. Publish Artifacts
4. Integration Tests
M
F RF D
CI
M
F RF D
CI
Deploy & TestDeploy & Test
Deploy to
Production
Deploy to
Production
Repo 2Repo 1
Deploy to Acceptance Environment & Test

REQUIREMENTS
1. Configuration as Code
2. Multibranch Coverage
3. Visibility
4. Reuse Job Configuration (D.R.Y)
5. Integration Tests Isolation
6. Self Contained Configuration

REQUIREMENTS

HOW

CODE REUSE - GLOBAL PIPELINE LIBRARIES
Jenkinsfile
#!groovy
@Library('buildtools@1.x') _
properties([
parameters([forcePublishParam()]),
discardOldBuildsProperty()
])
withMavenDockerNode(jdk: 'JDK8') { buildTools ->
stage('Setup') {
buildTools.init()
dir('app') {
checkout scm
}
}
dir('app') {
mavenBuildTestPublish()
}
failAsUnstable {
withDockerEx { dockerEx ->
def kafkaIp
stage('Start Sidecars') {
kafkaIp = startKafkaSidecar(dockerEx)
}
stage('Integration Tests') {
dir('app') {
runIntegrationTests(kafkaIp)
} } } } }
def call() {
stage('Build') {
sh 'mvn clean verify –U –B –fae –Dmaven.test.failure.ignore'
}
stage('Unit Tests') {
junit '**/target/surefire-reports/TEST-*.xml'
}
stage('Publish') {
def msg = canDeploy()
if (msg == 'ok' || params.FORCE_PUBLISH) {
sh 'mvn deploy'
} else {
echo "Publish skipped, cannot deploy: ${msg}"
}
}
}
vars/mavenBuildTestPublish

ISOLATION – “SIDECAR” PATTERN
Jenkins
Master
Jenkins
Slave
Jenkins
Slave
Kafka
Broker
Schema
Registry
Zookeeper
kafka sidecar
Kafka
Broker
Schema
Registry
Zookeeper
kafka sidecar

Docker Extension – Support Docker Compose
import com.ebay.tools.DockerEx
def call(body) {
def dockerEx = new DockerEx(this)
try {
body(dockerEx)
} finally {
dockerEx.close()
}
}
vars/withDockerEx
def call(dockerEx) {
def kafka = dockerEx.compose('kafka',
[file: 'buildtools/docker/kafka.yml'])
kafka.defaultNetwork.create()
def ip = kafka.inspectGateway()
echo "Kafka IP: ${ip}"
withEnv(["KAFKA_IP=${ip}"]) {
kafka.up().ps()
}
waitForPort host: ip, port: 29092 // Kafka broker
waitForEndpoint url: "http://${ip}:8081" // Schema Registry
ip
}
vars/startKafkaSidecar
withDockerEx { dockerEx ->
def kafkaIp
stage('Start Sidecars') {
kafkaIp = startKafkaSidecar(dockerEx)
}
stage('Integration Tests') {
dir('app') {
runIntegrationTests(kafkaIp)
}
}
}
class DockerEx implements Serializable {
protected final def script
private final ArrayMap compositions = new ArrayMap()
DockerEx(script) {
this.script = script
}
DockerExCompose compose(String name, Map config = [:]) {
DockerEx dockerEx = this
compositions.get(name, { ->
new DockerExCompose(docker, name, config)
}
}
void close() {
closeAllCompositions()
}
void closeAllCompositions() {
compositions.forEach { _, compose ->
try { compose.down() } catch (e) {
script.echo "Destroy '${compose.name}' failed: ${e}"
}
}
}
}
src/com/ebay/tools/DockerEx

Docker Extension – Support Docker Compose
class DockerExCompose implements Serializable {
private final DockerEx dockerEx
private final def script
private final ArrayMap config
final String name
DockerExCompose(DockerEx dockerEx, String name,
Map config = [:]) {
this.dockerEx = dockerEx
this.script = dockerEx.script
this.name = name
this.config = config
}
DockerExCompose up() {
script.sh "docker-compose –project-name $name " +
"${fileArg()} up -d"
this
}
DockerExCompose down() {
script.sh "docker-compose ${fileArg()} down"
this
}
src/com/ebay/tools/DockerExCompose
DockerExCompose ps() {
script.sh "docker-compose ${fileArg()} ps"
this
}
String inspectGateway() {
defaultNetwork.inspectGateway()
}
DockerExNetwork getDefaultNetwork() {
docker.network("${name}_default".toString())
}
private String fileArg() {
def file = config.get('file')
file ? "--file ${file}" : ''
}
}
String inspectGateway() {
def res = script.sh(script: "docker network inspect -f " +
"'{{range .IPAM.Config}}{{.Gateway}}{{end}}' $name",
returnStdout: true)
res.trim().split('/')[0].toString()
}

SELF CONTAINED CONFIGURATION
withMavenDockerNode
import com.ebay.tools.BuildTools
def call(Map args, body) {
timestamps {
def timeoutArgs = args.timeout ?: [time: 1, unit: 'HOURS']
timeout (timeoutArgs) {
def label = 'maven-docker-builder'
mavenDockerPodTemplate(label) {
node(label) {
def buildTools = new BuildTools(this, args)
body(buildTools)
}
}
}
}
}
private void mavenDockerPodTemplate(String label, body) {
podTemplate(
label: label,
containers: [
containerTemplate(
name: 'jnlp',
image: ...
)] ...
) {
body()
}
}
Jenkinsfile
#!groovy
properties([
parameters([forcePublishParam()]),
discardOldBuildsProperty()
])
...
src/com/ebay/tools/BuildTools
class BuildTools implements Serializable {
private final def script
private final ArrayMap args
void init() {
gitCloneBuildTools()
setJavaHome(args.get('jdk'))
printToolsVersion()
}
private void printToolsVersion() {
script.sh script: 'mvn --version', returnStatus: true
script.sh script: 'java -version', returnStatus: true
script.sh script: 'docker version', returnStatus: true
}
}

TIPS & TRICKS
● Do not load global libraries implicitly by default.
This is sometimes redundant and it is not visible
● Do load global libraries explicitly with specific stable development branch, e.g. “1.x”.
This way you get fixes and enhancements transparently, but keep the option for “2.x”
● Do treat your global pipeline library as any other library you write (e.g. versioning,
testing, etc.)
Global Pipeline Library Management

TIPS & TRICKS
Global Pipeline Library Management - Testing
#!groovy
import static com.ebay.tools.test.PipelineTestUtils.*
withMavenDockerNode {
pipelineTests(this) {
testSuite('withBuildTools, shEx') {
test("withBuildTools - Test build tools with JDK8") {
withBuildTools(jdk: 'JDK8') { buildTools ->
def result = shEx(script: 'java -version')
assertStringContains expected: 'version "1.8.',
actual: result.get('err'),
message: 'Java version not as expected'
assertEquals expected: 0,
actual: result.get('status'),
message: "Return status not as expected"
}
}
...
}
}
}

TIPS & TRICKS
● Do prototype, test and play using the online script editor
● Do use the “Replay” functionality
● Do echo, echo, echo – write decisions and state to the console
● Trick: interactiveShell
Pipeline Script Development and Debugging
def call() {
timeout(time: 10, unit: 'MINUTES') {
echo "[Shell] ***** Starting interactive shell *****"
String cmd = ''
while (cmd != 'exit') {
cmd = input(id: 'cmd', message: 'Command:', parameters: [
[$class: 'TextParameterDefinition', name: 'cmd',
description: 'Enter shell command to run, "exit" to stop the shell']
])
def ret = sh(script: cmd, returnStatus: true)
echo "[Shell] Return Code: $ret"
}
echo "[Shell] Bye!"
}
}

TIPS & TRICKS
● Do organize in stages to reflect your pipeline flow, think visibility
● Do echo, echo, echo (yes, I already wrote it, it’s important)
● Do control the result status to reflect the actual result
● Do write your global library to express your DSL
Increase Visibility and Readability

Any Questions?
THANK YOU!
Yinon Avraham
yavraham@ebay.com
@yinonavraham

Taking Jenkins Pipeline to the Extreme

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Taking Jenkins Pipeline to the Extreme

Similar to Taking Jenkins Pipeline to the Extreme (20)

Recently uploaded

Recently uploaded (20)

Taking Jenkins Pipeline to the Extreme

Editor's Notes