Automate CI/CD with Rancher

CI/CD with Rancher
How we automate container orchestration in our
environments at Kloeckner-i using Gitlab, Docker and
Rancher
Original google slides version: https://goo.gl/YiMQkJ

Nick Thomas
Developer
Code Fairy
Working at Kloeckner-i
github.com/pangdudu
nick.thomas@kloeckner.com

Topics
I. Introduction
II. Container Orchestration
III. CI/CD with Rancher
IV. Setup storage management in
Rancher
Introduction / Orchestration / CI/CD with Rancher / Storage management in Rancher
source:medium.com

Kloeckner-i
● Kloeckner-i, a corporate-startup
hybrid based in Berlin
● We’re building the next
generation of tools for the steel
industry
● To spend as much time as
possible developing, we need to
automate everything else
source:pdssheetmetal.co.uk

Automate all the things!
What we want to automate:
● Checking our code style
● Testing our code
● Creating QA environments and
deploying our code to them
● Running integration tests
● Deploying our code to production
systems
● Scaling our systems if we need
to

What we’re building - Microservices
Platform
Frontend
frontend.DOMAIN
...
API Proxy
api.DOMAIN
Backend Magic
(Services)
ERP (SAP)
IDP
IdentityProvider
idp.DOMAIN
Account Producer
Contract Producer
Order Producer
ConnectorAPI
Central processing system
(Multiple queues)
Consumer
Consumer
Web
frontend.DOMAIN

Container Orchestration
● Tools that organizations use to manage
containers at scale (and in production)
● May include features such as container:
○ Upgrades
○ Scaling
○ Multi Host Networking
○ Storage management
○ Health checks
○ ...
source:quora.com

So why Rancher?
Candidates
● Amazon ECS
● Google Container Engine
● Azure Container Service
● Apache Mesos
● Kubernetes
● Docker Swarm
● CF Diego
● CoreOS Fleet
● Puppet, Bash, ...
Reasons I chose Rancher
● Wanted no vendor lock-in (self hostable
and open-source)
● Something developers knowing
docker-compose understand
● Docker Swarm (1.12) not ready
● Mesos and Kubernetes more complicated
● Github: 4K, Issues: open 1K, closed 4K
● Powerful GUI
● Just feels right

CI/CD with Gitlab, Docker, Rancher
Current stack
● Hosting: Google Compute Engine
● Automation: Gitlab
○ Run tests: docker runner
○ Host docker images: docker registry
● Orchestration: Rancher
source:lakehub.co.ke

Docker development workflow
1. Build Dockerfile
2. Run container
3. Build docker-compose.yml
4. Run container with dependencies

Gitlab Dockerception
1. Build .gitlab-ci.yml
a. Lint
b. Test
c. Build docker image
2. Push to repository
3. Gitlab runs your jobs
4. Build docker containers within
docker containers with docker

.gitlab-ci.yml

Gitlab Dockerception - Demo

Rancher - Another layer on top
Source:rancher.com

Rancher Features
● Rancher structure
○ Environments
○ Stacks and Services
○ Catalog
● Discoverable API
● Easy to setup
Source:rancher.com

Rancher - Demo

SHIA - Just do it!
● Connecting Gitlab and Rancher
● Rubygem SHIA
● Deploy stack to environment
● Deploy full stack to a “QA” environment
● Don’t let your dreams be dreams …
●
SHIA stands for Seamless Hive Integration Agent (it’s really hard to find buzzwords for that acronym )

SHIA - Deploying Services
● Create Environment
● Add hosts on GCE via docker-machine
● Add users to environment
● Process stack config
● Create description (branch:commit-hash)
● Create or update stack
● One command: shia -e linuxcon deploy

SHIA - Deploying QA Environments
● Convention “@env_xx” in git branch name
● Deploys stack that is to be tested with current branch
● Deploys the rest of the stack with master branch
● Multiple stacks can be deployed to the same environment with different
branches
● One command: shia -b ${CI_BUILD_REF_NAME} deploy_all

SHIA - Deploying Services - Demo

SHIA - Deploying QA Environments - Demo

SHIA - What we’re working on
● Automate GCE networking configuration
○ Create GCE LB
○ Configure DNS (currently using myra cloud)
● Seed data efficiently in QA environments
● Run full stack integration tests in QA environments

Storage Management with
Rancher

Storage options with Docker
● Default : No Data Persistence
● Data Volumes : Container Persistence
● Data Only Container : Container
Persistence
● Host Mapped Volume : Container
Persistence
● Host Mapped Volume, backed by Shared
Storage : Host Persistence
● Convoy Volume Plugin : Host Persistence
(of Storage Host)

Convoy-NFS features
● Docker Volume Driver Plugin
● Backend for different storage options
○ Device Mapper
○ NFS/EFS/VFS
● Snapshots, Backups, Restore
● Single Go binary
● Can be deployed independently of Rancher

Convoy-NFS
Source:rancher.com

Setting up convoy-nfs

Where to get SHIA
Kloeckner-i Github
https://github.com/kloeckner-i/shia
Introduction / Orchestration / CI/CD with Rancher / Storage management in Rancher / Fin

Introduction / Orchestration / CI/CD with Rancher / Storage management in Rancher / Fin
It’s easy. JUST DO IT!

Original google slides version: https://goo.gl/YiMQkJ

Automate CI/CD with Rancher

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Automate CI/CD with Rancher

Similar to Automate CI/CD with Rancher (20)

Recently uploaded

Recently uploaded (20)

Automate CI/CD with Rancher

Editor's Notes