More Related Content Similar to DevOpsCon London: How containerized Pipelines can boost your CI/CD (20) More from Nico Meisenzahl (13) DevOpsCon London: How containerized Pipelines can boost your CI/CD2. Nico Meisenzahl
• Senior Cloud & DevOps Consultant at white duck
• GitLab Hero, Microsoft MVP & Docker Community
Leader
• loves Kubernetes, Cloud-Native & DevOps
© white duck GmbH 2020
Phone: +49 8031 230159 0
Email: nico.meisenzahl@whiteduck.de
Twitter: @nmeisenzahl
LinkedIn: https://www.linkedin.com/in/nicomeisenzahl
Blog: https://meisenzahl.org
3. Agenda
• containerized pipeline – why?
• pipeline workload on Kubernetes with GitLab CI/CD
• container image builds on Kubernetes with Kaniko
• cloud-native pipelines with Tekton
© white duck GmbH 2020
4. Containerized Pipelines – Why?
• for the same reasons why you should use containers
• isolation
• dependencies
• scalability
• immutability
• example: your new project needs version X all others still require Y
• you can include any kind of build / deploy dependency
• NodeJS, .NET Core, Go, Terraform, Ansible… you name it
© white duck GmbH 2020
5. docker run
• running Docker on build hosts is a familiar pattern
• but: docker run pipeline jobs have some issues
• containers are immutable
• you will end up managing caching and artifacts on your own
• (and we're not even talking about building containers)
© white duck GmbH 2020
6. GitLab Runner Kubernetes executor
• a Gitlab Runner that is aware of Kubernetes
• integrates your CI/CD with Kubernetes
• runs itself within your Kubernetes Cluster (as pod)
• schedules pipeline jobs as pods
• allows you to share your compute and scale your pipelines
• easy deployment
• one-click
• production ready Helm deployment
© white duck GmbH 2020
7. Cloning, Caching and Artifacts
• GitLab Runner Kubernetes executor runs service containers within
the pipeline job pod to handle cloning, caching and artifacts
• every pipeline job contains out of these stages
• prepare → creates pod with build and service containers
• pre-build → clones repo, restore cache, download artifacts
• build → user build steps
• post-build → creates caches and upload artifacts
© white duck GmbH 2020
8. Advanced topics
• supports advanced configurations like
• resource limits for build jobs
• CI services for build dependencies
• node selectors for runners
• …
© white duck GmbH 2020
9. Pipeline job image
• contains everything a single pipeline job needs
• binaries, libraries, tools, ...
• provides all external dependencies
• uses a pipeline to build/rebuild it periodically (security fixes!)
• you should define fix versions for your dependencies
• uses caching to speed up your builds
© white duck GmbH 2020
12. Container Image builds on Kubernetes
• you may use Docker-in-Docker to build containers in containers
• DinD has some issues
• exposing Docker socket
• mounting /var/lib/docker
• privileged mode
• those might be acceptable on a build host but not in a Kubernetes
cluster
© white duck GmbH 2020
13. Image builds with Kaniko
• image builds without the need of any privileges or dependencies
• runs in a container (gcr.io/kaniko-project/executor)
• easy to use
• you should think about caching to speed up your pipeline
• layer caching (layers get pushed to a registry)
• base image caching (GitLab cache)
© white duck GmbH 2020
15. Demo
• containerized image builds on Kubernetes with Kaniko
• https://gitlab.com/containerized-cicd/containerized-jobimage
© white duck GmbH 2020
16. Cloud-native pipelines with Tekton
• Tekton Pipelines emerged out of the Knative Build project
• founding Continuous Delivery Fundation project
• „CI/CD framework for Kubernetes”
• Jenkins X pipelines are based on Tekton
• contributions by Google, IBM, RedHat, Cloudbees, TriggerMesh, …
© white duck GmbH 2020
17. Tekton Pipelines are …
• … Cloud Native
• run on Kubernetes
• use containers as their building blocks
• managed with Controllers
• … decoupled
• a pipeline can be reused in multiple clusters
• tasks can be isolated and are reusable
• resources can be swapped between runs
© white duck GmbH 2020
18. Triggers, CLI & Dashboard
• Tekton Triggers can be used to call a pipeline
• push, issue, webhook, ….
• Tekton also provides a CLI and Dashboard
© white duck GmbH 2020