Clocker - How to Train your Docker Cloud
•
0 likes•693 views
Talk at QCon London, Docker Track, March 2015
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Clocker - How to Train your Docker Cloud
Similar to Clocker - How to Train your Docker Cloud (20)
More from Andrew Kennedy
More from Andrew Kennedy (15)
Recently uploaded
Recently uploaded (20)
Clocker - How to Train your Docker Cloud
- 1. QCon London 2015; Docker Track Andrew Kennedy @grkvlt Clocker How to Train Your Docker Cloud
- 2. @grkvlt Introduc/on • Andrew Kennedy – Clocker Project Founder and Lead Engineer – Open Source and Distributed Systems – github.com/grkvlt • CloudsoG Corpora/on – Open Source Applica/on Management Specialists – We’re Hiring…
- 3. @grkvlt Agenda • Clocker Introduc/on • What is a Docker Cloud? • Training a Docker Cloud • Demonstra/on • Roadmap
- 6. @grkvlt What does it do? 1. Spins up and Manages Docker Clouds 2. Serves up Containers on Demand 3. Manages Composite Applica/on Deployments on Docker
- 7. @grkvlt What does it provide? • Mul/ Host and Mul/ Container Applica/ons • Seamless Networking – Communica/on Between Services • Orchestra/on and Clustering – Control of Containers – Container Management
- 8. @grkvlt Who is using it? • Tes/ng and Proof of Concept Stage – Financial Services – Insurance • Produc/on – Mul/-‐tenant Applica/on Trial – Container per service – Via IBM BlueMix
- 9. @grkvlt Where can I find it? • Open Source • Apache 2.0 Licensed • h[p://clocker.io • Status • 0.7.0 Beta Release • 0.8.0-‐SNAPSHOT Development
- 10. What is a Docker Cloud?
- 11. @grkvlt Clocker and Brooklyn • What is it? – Brooklyn Applica/on – Brooklyn Loca/on • What does it provide? – First Class Docker Support in Brooklyn – Op/mized Brooklyn Blueprints for Docker
- 12. Apache Brooklyn • Apache Brooklyn – Open Source – Java – Donated to the Apache SoGware Founda/on – ASF Incubator Status
- 13. @grkvlt Apache Brooklyn • Applica/on Management Pla_orm • Deploy, Manage and Monitor Blueprints • Provisioning, Installa/on and Customiza/on • Management – AutoScaling, Resilience, Performance, Security
- 14. Apache jclouds • Java Cloud Library • API Agnos/c – CloudStack, OpenStack, AWS EC2, GCE… • Create Virtual Machines – Return SSH Endpoint – Manage Proper/es
- 15. @grkvlt Apache jclouds • Drivers for REST APIs • Docker Driver – Wri[en by @turlinux • Virtual Container – Using SSH Daemon – Same Endpoint Type as VM – Composi/on on any Image or Dockerfile
- 16. @grkvlt Docker • Popular – Large Ecosystem – Growing – Complex • Containers – Isola/on – Performance – Composable
- 17. @grkvlt Docker
- 18. @grkvlt Brooklyn Blueprints • Describe Applica/ons • OASIS CAMP Standard • List of Services • Tree Structure • Sensors, Effectors and Policies
- 19. @grkvlt Blueprint Example name: appserver-‐w-‐policy services: -‐ type: brooklyn.entity.webapp.ControlledDynamicWebAppCluster initialSize: 1 memberSpec: $brooklyn:entitySpec: type: brooklyn.entity.webapp.jboss.JBoss7Server brooklyn.config: wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-‐example-‐hello-‐world-‐sql-‐webapp/0.6.0/brooklyn-‐example-‐hello-‐world-‐sql-‐ webapp-‐0.6.0.war http.port: 8080+ java.sysprops: brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s&password=%s", component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") brooklyn.policies: -‐ policyType: brooklyn.policy.autoscaling.AutoScalerPolicy brooklyn.config: metric: $brooklyn:sensor("brooklyn.entity.webapp.DynamicWebAppCluster", "webapp.reqs.perSec.windowed.perNode") metricLowerBound: 10 metricUpperBound: 100 minPoolSize: 1 maxPoolSize: 5 -‐ type: brooklyn.entity.database.mysql.MySqlNode id: db name: DB HelloWorld Visitors brooklyn.config: datastore.creation.script.url: https://github.com/apache/incubator-‐brooklyn/raw/master/usage/launcher/src/test/resources/visitors-‐creation-‐script.sql
- 20. @grkvlt Applica/on Management • Policies • Sensor Data Driven • Auto Scaling • Cluster Management • Resilience and Failover • Follow the Sun
- 21. @grkvlt Docker Cloud • Brooklyn Blueprint for Docker Cluster • Docker Engine on Cloud VM – Configura/on for TLS Cer/ficates – Setup Volumes (Logging?) – Install Agents (Security? SDN?)
- 22. @grkvlt Docker Cloud 1. On-‐demand 2. Mul/-‐Tenant 3. Hardware Independent 4. Applica/on Level
- 24. @grkvlt Clocker Features • Applica/on Deployment – Oasis CAMP YAML Blueprint – TOSCA in Development – Same as Core Brooklyn • Mixed Des/na/ons – Some Virtual Machines – Some Bare Metal – Some Containers
- 25. @grkvlt Clocker Features • Docker Extensions to Brooklyn – Dockerfile or Image Specifica/on for Installa/on – Placement Strategies for Containers – Create Docker Images and Networks • Manages Docker Engine – Deployment and Management – Installa/on and Configura/on – SoGware-‐Defined Networking
- 26. @grkvlt Clocker Orchestra/on Docker Engine Virtual Machine Container Clocker Network Segment SDN Cloud
- 27. Training a Docker Cloud
- 28. @grkvlt Training…? • Container Management • Placement and Provisioning • Autonomics – Policies and Scaling • Headroom • Networks
- 29. @grkvlt Container Management • Sources – Docker Image Defini/on – Docker Hub – Dockerfile – Brooklyn En/ty Defini/on • Create Image Automa/cally – Commit or Push for Reuse
- 30. @grkvlt Container Management • Installa/on of Services – Defined by Brooklyn or Dockerfile – Common to all En/ty Instances • Commit Image – Available for next En/ty • Push Image – Available for all Hosts
- 31. @grkvlt Container Management id: dockerfile-‐mysql name: "Docker Hub MySQL Database" origin: "https://registry.hub.docker.com/_/mysql/" locations: -‐ my-‐docker-‐cloud services: -‐ type: brooklyn.entity.container.docker.application.DockerfileApplication id: mysql name: "MySQL" brooklyn.config: docker.dockerfile.url: "https://s3-‐eu-‐west-‐1.amazonaws.com/brooklyn-‐clocker/mysql-‐5.6.tgz" docker.container.environment: MYSQL_ROOT_PASSWORD: "s3cr3t”
- 32. @grkvlt Container Management id: dockerfile-‐mysql name: "Docker Hub LAMP Stack" locations: -‐ my-‐docker-‐cloud services: -‐ type: docker:mysql:5.7.5 id: mysql env: MYSQL_ROOT_PASSWORD: "s3cr3t" -‐ type: docker:grkvlt/myapp:latest id: application env: MYSQL_HOST: component(”mysql").attributeWhenReady("host.hostname")
- 33. @grkvlt Container Placement • Demand – Adding an Applica/on – Scaling exis/ng Applica/on • Requirements – Host Loca/on – Service Resources – CPU, Memory
- 34. @grkvlt Container Placement • Supply – Choose a Host from available – Create new Host • Start Container – Set CPU and Memory – A[ach to Network
- 35. @grkvlt Container Placement • Placement Strategies – Filter Hosts – Sort Hosts • Informa/on from – Docker – Underlying Machine – Cloud Environment – En/ty or Service
- 36. @grkvlt Container Placement • Placement Strategies – Random, Depth or Breadth First – CPU or Memory Usage – Memory, CPU or Container Limits – Geographic Constraints • User Defined – Java Predicate
- 37. @grkvlt Placement Strategy • Determinis/c • Simple – Predicate and Comparator docker.container.strategies: -‐ $brooklyn:object: type: "brooklyn.location.docker.strategy.MaxContainersPlacementStrategy” brooklyn.config: maxContainers: 16 -‐ $brooklyn:object: type: "brooklyn.location.docker.strategy.CpuUsagePlacementStrategy” brooklyn.config: maxCpu: 0.75
- 38. @grkvlt Container Provisioning • Provisioning Strategy – New Docker Host Loca/on • Constraints – Docker Infrastructure Constraints – En/ty or Applica/on Constraints • User Defined Strategies • Intelligent Container Orchestra/on
- 39. @grkvlt Autonomics • Brooklyn Policies • A[ached to En//es in Applica/on – Nothing Docker Specific • Elas/c Scaling – Cluster Resizing – Sensor Driven
- 40. @grkvlt Applica/on Resilience • Service Resilience and Replacement – Restart Service and Container – Applica/on Level, Not Infrastructure – Same as Cloud • Could be Integrated Further – Snapshot Running Container for Restart
- 41. @grkvlt Headroom • Ensure resources available • Based on MaxContainers strategy limit – Or Percentage U/lisa/on – Or CPU and RAM alloca/on • Scale Docker Host Cluster Automa/cally – Add new Docker hosts – Remove empty Docker hosts
- 42. @grkvlt SoGware-‐Defined Networking • Needed for Seamless Provisioning • Host to Host Communica/on – Same LAN Segment – No Port Forwarding – Natural Applica/on Configura/on • Ini/al Driver was EPMD Applica/ons
- 43. @grkvlt Networking Providers • Standardized Interfaces • Pluggable Providers – Weave – Metaswitch Calico – IBM OpenDOVE – Others?
- 44. @grkvlt Networking Providers • Implementa/on Agnos/c – L2 over L3 etc. – Similar to Hypervisor in Clouds • Generic Interfaces – Host Component – Service Component (or Endpoint)
- 45. @grkvlt Networking Capabili/es • Provide Mul/ple Networks – Single Applica/on or Shared – Private Addresses – Segmented by CIDR • Docker Port Forwarding Access – Debug Mechanism
- 46. @grkvlt Clocker Networking Container Host SDN Bridge Container Internet SDN Gateway
- 47. @grkvlt Networking Capabili/es • IP Pool Controlled by Clocker • Clocker Controls Subnet Alloca/on – Applica/ons Segmented by CIDR • Delegate to SDN or Cloud – Bring your own IP • Docker Port Forwarding Access
- 48. @grkvlt Future Capabili/es • Adding More SDN Providers • Name Resolu/on – BIND and DNSmasq or Weave DNS – Needed for JMX et al – Enables Many More En//es • Adding More Supported Blueprints
- 49. @grkvlt Virtual Network En/ty • New Clocker En/ty • Provisions a Virtual Network • Looks for NetworkProvisioningExtension – Registered in DockerLoca/on by SDN Provider – Could also add to JcloudsLoca/on for Clouds • Part of Applica/on Blueprint
- 50. @grkvlt Network Provisioning • Create Named Virtual Networks • Uses Available Provider – Currently Weave, Calico and IBM Supported -‐ type: brooklyn.networking.VirtualNetwork networkId: my-‐application cidr: 192.168.12.0/24 gateway: 192.168.12.1 dnsServers: -‐ 8.8.8.8 brooklyn.config: sdn.example.securityGroup: "my-‐security-‐group"
- 51. @grkvlt Network Provisioning • Allow Minimal (Zero!) Configura/on • Use Sensible Defaults – Also SDN Specific Configura/on • Allocate Address Space on Demand -‐ type: brooklyn.networking.VirtualNetwork networkId: database-‐net cidr: 192.168.34.0/24 -‐ type: brooklyn.networking.VirtualNetwork networkId: couchbase-‐net
- 52. @grkvlt Network Provisioning • A[ach Containers to Networks brooklyn.config: network.list: -‐ couchbase-‐net -‐ management-‐net • Create Networks as Required • Also A[ach to VMs and Metal – Where Supported
- 53. DemonstraBon
- 58. @grkvlt Features • Orchestrated Docker 1.5.0 deployment with Weave SDN integra/on • Automated a[achment of containers to mul/ple dynamic networks • Brooklyn applica/on blueprints with network topology
- 59. Clocker Roadmap
- 60. @grkvlt Roadmap • Latest Docker 1.5.0 Feature Support – Stats command, Read-‐only containers, Named Dockerfiles and more • Swarm – Add Clocker as an extension to Swarm itself? – Or add features to exis/ng jclouds driver? • Compose – Support as alterna/ve blueprin/ng language
- 61. @grkvlt Roadmap • Networking – DNS and DNSmasq Integra/on – Rancher Integra/on – Open vSwitch Integra/on • Gemng Started – Clocker Dockerfile
- 62. @grkvlt Roadmap • Repositories – Docker Enterprise Hub – Bintray, Ar/factory, Quay.io – Private Repositories • Applica/on Defini/on – TOSCA – Docker Compose
- 63. @grkvlt Roadmap • Plugin API will be helpful • Also projects like Powerstrip • Provide services to Docker this way • Orchestra/on • Policies • Consume wider range of services • Networking and Storage
- 64. @grkvlt Summary • Clocker – Brooklyn + jclouds + Docker + Weave + … – @clockercentral • Solves – Composite Applica/on Management – Docker Cloud Networking – Container Placement and Provisioning
- 65. QuesBons?
- 66. @grkvlt Web Resources http://clocker.io/ http://brooklyn.io/ http://docker.io/ http://github.com/zettio/weave/ http://abstractvisitorpattern.co.uk/ @clockercentral @brooklyncentral @cloudsoft