Portable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins
•
0 likes•147 views
How to establish Kubernetes as your infrastructure for a truly cloud native environment for optimal productivity and cost. Using Kublr for infrastructure as code approach for fast, reliable and inexpensive production-ready DevOps environment setup bringing together a combination of technologies - Kubernetes; AWS Mixed Instance Policies, Spot Instances and availability zones; AWS EFS; Nexus and Jenkins. Best practices based on open source tools such as Nexus and Jenkins. How to tackle build process dilemmas and difficulties including managing dependencies, hermetic builds and build scripts.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Portable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins
Similar to Portable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins (20)
More from Kublr
More from Kublr (12)
Recently uploaded
Recently uploaded (20)
Portable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins
- 1. Portable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins
- 2. Introductions Oleg Chunikhin CTO, Kublr • 25 years in software architecture & development • Working w/ Kubernetes since its release in 2015 • Software architect behind Kublr—an enterprise ready container management platform • Twitter @olgch
- 3. Click to add text Click to add text Learn more at kublr.com/how-it-works @kublr, @olgch
- 4. Kublr. Turnkey Container Management ✔ Rollout production-ready enterprise container platforms in a matter of days ✔ Automate the deployment and configuration of Kubernetes and all necessary components. ✔ Built on top of open source Kubernetes. ✔ Provides the operational and security features enterprises require. @kublr, @olgch
- 5. •Demo project: https://github.com/kublr/devops-demo-v2 •DevOps Architecture •Kubernetes as Application management framework •Cloud Native CI/CD Architecture •CI/CD Demo Deployment: self-managed, portable, reliable •Everything-as-Code in examples Topics @kublr, @olgch
- 6. DevOps Architecture @kublr, @olgch Infrastructure VMs Network Storage Ingress Load Balancers Dev / CI / CD Tools VCS CD CI Binary Repos Project Structure Source Code Dev / Release Process Binaries Ops Tools IDM Logging Monitoring Provisioning Frameworks Containers Serverless Service Mesh App Servers
- 7. Everything-as-Code @kublr, @olgch Infrastructure VMs Network Storage Ingress Load Balancers Dev / CI / CD Tools VCS CD CI Binary Repos Project Structure Source Code Dev / Release Process Binaries Ops Tools IDM Logging Monitoring Provisioning Frameworks Containers Serverless Service Mesh App Servers
- 8. Kubernetes Kubernetes @kublr, @olgch Infrastructure VMs Network Storage Ingress Load Balancers Dev / CI / CD Tools VCS CD CI Binary Repos Project Structure Source Code Dev / Release Process Binaries Ops Tools IDM Logging Monitoring Provisioning Frameworks Containers Serverless Service Mesh App Servers Kubernetes = App ops framework + Application abstraction + Infrastructure abstraction
- 9. Kubernetes Tools Variety @kublr, @olgch Infrastructure VMs Network Storage Ingress Load Balancers Dev / CI / CD Tools VCS CD CI Binary Repos Project Structure Source Code Dev / Release Process Binaries Ops Tools IDM Logging Monitoring Provisioning Frameworks Containers Serverless Service Mesh App Servers Kubernetes = App ops framework + Application abstraction + Infrastructure abstraction
- 10. Kubernetes Demo @kublr, @olgch Infrastructure VMs Network Storage Ingress Load Balancers Dev / CI / CD Tools VCS CD CI Binary Repos Project Structure Source Code Dev / Release Process Binaries Ops Tools IDM Logging Monitoring Provisioning Frameworks Containers Serverless Service Mesh App Servers Kubernetes = App ops framework + Application abstraction + Infrastructure abstraction
- 11. Registry Monitoring Prod Target CI/CD Architecture CI CD QA Env Build Run Unit Tests Build Docker Image Push Docker Image Deploy to QA K8s Run Tests Monitor Deploy to Prod Manage Traffic Monitor Monitoring Docker Registry Prod Cluster QA Cluster @kublr, @olgch Git Git Repository Dev Env QA Cluster DevOps Env
- 12. GitHub Git Repository Nexus Monitoring Prod Target Cloud Native CI/CD Architecture Jenkins Spinnaker QA Env Build Run Unit Tests Build Docker Image Push Docker Image Deploy to QA K8s Run Tests Monitor Deploy to Prod Manage Traffic Monitor Monitoring Docker Registry Prod Cluster QA Cluster @kublr, @olgch Dev Env Dev Cluster DevOps Env
- 13. Nexus Demo Environment Jenkins Build Run Unit Tests Build Docker Image Push Docker Image Docker Registry @kublr, @olgch GitHub Git Repository Dev Env QA Cluster DevOps Env Jenkins Agent(s) Agent
- 14. •Infrastructure: Kublr K8S on Azure and AWS • Kublr K8S, cluster spec • Kubernetes: multi-master • AWS • Multi-zone • Mixed Instance Policy, spot • EFS + K8S NFS Provisioner • Elastic LoadBalancer + Route 53 • Nginx Ingress Controller • Letsencrypt + K8S Certmanager Infrastructure Highlights @kublr, @olgch • Azure • Managed Disks + K8S Azure Cloud Provider • Azure LoadBalancer + Route 53
- 15. •Dev/CI/CD Tools • GitHub • CI • self-managed Jenkins • Helm deployment, included in the cluster spec • Registry • self-managed Nexus • Helm deployment, included in the cluster spec Dev/CI/CD Tools Highlights @kublr, @olgch
- 16. •Tools configuration scripts: hacks/ • Kublr: get cluster kubeconfig • Nexus: create docker registry, enable anonymous login • Jenkins: build params, build job, multi-branch •Project Build Automation • Build pipeline: Jenkinsfile • Build: simple-api/Dockerfile • Packaging: simple-api/Dockerfile • Packaging: simple-api/helm/ • Deployment: simple-api/helm/ Code Highlights @kublr, @olgch
- 17. •Version: main.properties •Branches: release (main, release/*) and non-release •Artifact tag: builder/tag.sh • Release: 0.0.1 • Non-release: 0.0.1-featureblue.25.1dfe0deb.dirty •Deploy: namespace per branch Branching and Release Process @kublr, @olgch
- 18. • Allow Anonymous Access • Security > Realms: add DockerBearerToken realm • Create and configure docker hosted registry • port HTTP 5003 • allow anonymous pull • disable redeploy (allow ‘latest’ tag update) Nexus Setup @kublr, @olgch
- 19. • Create a multi-branch pipeline build project • define parameters and credentials (docker repo and ingress domain) • Create a multi-branch pipeline job • register git repo (add “Checkout local branch” action) Jenkins Project Setup @kublr, @olgch
- 20. •Provisioning and operations: • Kublr 1.20+ •Infrastructure: • AWS or Azure • Multi-zone / multi-master • AWS: Spot instances, 3 x t3.medium, 3 x m5a.large • AWS: Mixed Instance Policy with multiple instance types DevOps Environment K8S Cluster @kublr, @olgch
- 21. • Cloud Native Storage • AWS EFS • NFS Provisioner • Ingress • Route 53 • Nginx Ingress • Certmanager • Letsencrypt DevOps Environment K8S Cluster @kublr, @olgch • Cloud Native Storage • Azure Managed Disks • Kubernetes Azure Cloud Provider • Ingress • Route 53 • Nginx Ingress • Certmanager • Letsencrypt
- 22. •Cluster mgmt UI vs cluster spec •Cloudformation, standard and extra resources •AWS EFS and private Route 53 zone for EFS •AMI, Instance Type, ASG, MIP and Launch Template customization •Spot instances •Customization to use EFS storage for Kublr monitoring data •Embedded Helm packages: NFS provisioner, Jenkins, Nexus AWS: Cluster Spec Highlights @kublr, @olgch
- 23. • Hermetic build with composite Dockerfile • Docker image is the build artifact •Helm chart is used for deployment (helm create ...) •“Jenkinsfile” fully defines Jenkins pipeline •Release and non-release branches and image tags • Every branch is built and deployed in a separate namespace Project Structure @kublr, @olgch
- 24. • Infrastructure: cluster spec • Ops Tools: cluster spec • Dev/CI/CD Tools: cluster spec, scripts • Docker Image: source code, Makefile, Dockerfile • Build/Deploy Pipelines: source code, Jenkinsfile Everything as Code @kublr, @olgch
- 25. • Monorepo vs multiple repos • Managing dependencies • Hermetic builds: what about local development? • Build image: composite vs elementary build images • Build scripts: DRY is difficult with Jenkins vs local build • Agent pods: reusable vs one-time Build Process Dilemmas and Difficulties @kublr, @olgch
- 26. • K8S Resource management and build optimization • Feature namespace cleanup • Separate DevOps an Dev cluster • CD / Spinnaker / Canary releases • Dev and Release binary repositories • Helm chart release and repositories Beyond Simple Demo @kublr, @olgch • Private registries • git tagging on release • gitops • Secret and parameters management • Security hardening
- 27. Oleg Chunikhin | CTO oleg@kublr.com @olgch Q&A Sign up for our newsletter on kublr.com and stay in touch!