Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Loadbalancers: The fabric for your micro services

Presented at API World 2016, San Jose, CA

  • Be the first to comment

  • Be the first to like this

Loadbalancers: The fabric for your micro services

  1. 1. LOADBALANCERS: THE FABRIC FOR YOUR MICROSERVICES Chiradeep Vittal API World 2016
  2. 2. About me ¨ Distinguished Engineer at Citrix ¨ Work on Citrix Netscaler and containers
  3. 3. Load balancers are important (again) ¨ Containers and Microservices ¨ Background on Load Balancing ¨ Teaching old load balancers new tricks ¨ Emerging patterns for Load Balancing in microservices ¨ Future directions
  4. 4. Information Technology is rapidly accelerating Time Web Digital Social Mobile Things Smart Things RateofChange
  5. 5. The challenge for IT Match the velocity of business change
  6. 6. Addressing the gap: Agile © Christopher Little / Wikimedia Commons / CC-BY-3.0
  7. 7. ¨ Collaboration between Dev and Ops teams ¨ Tooling and automation to squeeze commit-to-deploy time Addressing the Gap: DevOps
  8. 8. Agile and DevOps 1. Frequent Iterations 2. All phases all the time 3. Risk-based 1. Shorten time from commit to production 2. “You build it, You run it” 3. Measure everything 4. Experimentation and learning Agile DevOps
  9. 9. Agile + DevOps Revolution 10+ Deploys per Day Flickr, 2009 50 million Deploys Per Year Amazon Web Services, 2015
  10. 10. Image by BMW Werk Leipzig - http://bmw-werk-leipzig.de, CC BY-SA 2.0 de, https://commons.wikimedia.org/w/index.php?curid=11928438 Key Mechanisms: Automation
  11. 11. Key Mechanisms: Microservices LB Web Data
  12. 12. 1. Small Autonomous processes 2. Loose coupling 3. Bounded contexts 4. Deploy & scale independently Key Mechanisms: Microservices
  13. 13. Containers & speed Compile (seconds) VM Build / Configure (minutes) Deploy VM (minutes) Compile (seconds) Container Build (seconds) Container Deploy (seconds) Before Containers (virtual machines) With Containers
  14. 14. Container Wins ¨ Small & Autonomous ¨ Deploy and scale independently ¨ Increased Velocity ¨ Reduced risk ¨ Efficiency Features Benefits
  15. 15. Containers or Microservices? ¨ Microservices == architecture ¨ Containers == implementation ¨ Containers win over VMs for Microservices
  16. 16. Monoliths vs. Microservices LB Web Data
  17. 17. Load Balancing: Traditional vs. Cloud Native Static Applications, well defined topology Dynamic Microservices, changing topology W W W A A A A A W M M TrafficismostlyN-S M M M M M M M M M Traffic is mix of N-S and E-W
  18. 18. Cloud-Native Patterns of architecture and organization that deliver software with speed and reliability ¨ Auto Scale ¨ Continuous Delivery ¨ Baked-in Resilience ¨ Deep Monitoring ¨ Collaboration
  19. 19. Cloud Native + Containers is Network Intensive ¨ Implications on ¤ Performance ¤ Reliability ¤ Security ¤ Routing ¤ Naming and discovery ¤ Monitoring
  20. 20. Cloud Native Landscape - Microservices Microservices APIs Performance Resilience Security Visibility Continuous Delivery AutoScal e Circuit Breaker Load Balancing Throttling Discovery Audit Segmentation E2E Encryption Routing Chaos Monkey Distributed Debug Back-off Lifecycle Management Auth
  21. 21. Containers Container Networking Container Security Integrity Isolation AuthN/Z Container Orchestration Infrastructure Orchestration Network Orchestration Cloud Orchestration Multi-cloud Orchestration Private Cloud OrchestrationHypervisor Orchestration Network Orchestration IPAM / DNS Overlay / Underlay Microservices Lifecycle Management Container Performance Network Perf scheduler Container Storage Storage Orchestration Storage Orchestration Cloud Native Landscape - Infrastructure
  22. 22. Container Managers to the rescue? Docker Swarm Kubernetes (from Google) Mesosphere DCOS (based on Apache Mesos) AWS ECS
  23. 23. Container Cluster Managers α1 β1 δ1 β2 α2 δ3 α3 α4 δ2 α5 α6 HostH1 HostH2 HostH3 HostH4 δ4 ClusterManager Cluster API
  24. 24. Cluster Managers – Sweet Spots ¨ Infrastructure independence ¨ Scheduling ¨ Discovery ¨ Scaling (partially) ¨ Security (a little bit) ¨ Load balancing (limited)
  25. 25. A brief history of Load balancing Internet W W High Availability Internet W W WW HA + Load Balancing Internet Webserver
  26. 26. A brief history of Load balancing Internet W WW App 1 App 2 W WW foo.com/app1 foo.com/app2 L7 routing Internet W WW SSL Traffic Unencrypted Traffic Internet W WW App Firewall
  27. 27. What LB-in-the-wire enables ¨ Resilience ¨ Encryption offload ¨ Application acceleration ¨ Defend L4à L7 ¨ L7 routing ¨ Application Performance Analytics
  28. 28. Load Balancing Form Factors Hardware Software VM Virtual As-a-service Containerized
  29. 29. Physical Price-Performance MPX VPX Hypervisor Virtual Run Anywhere = NetScaler CPX: Cloud Native Load Balancer CPX (new) Container SDX Platform Multi-Tenant =
  30. 30. NetScaler CPX Containerized NetScaler for Developers / Cloud native deployments ¨ Same hardened code, same features* as bigger form factors ¨ Unified control plane with Ingress and other LB (NetScaler MAS) ¨ Unified monitoring, logging and analytics * Same as NetScaler Standard Edition minus VPN
  31. 31. NetScaler CPX Express ¨ “Developer” Edition* ¨ No license required ¨ Available for download from Docker App Store in Q3 ¨ Drop-in replaceable with licensed version ¨ Get it here: – http://microloadbalancer.com *Not for production use
  32. 32. https://www.microloadbalancer.com
  33. 33. Load Balancing for container clusters – Ingress / Edge α5 α6 HostH4 Scale out Internet α1 β1 δ1 HostH1 β2 α2 δ3 HostH2 α3 α4 δ2HostH3 Public Endpoint https://alpha:443 LB
  34. 34. Load Balancing – intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LB LB per endpoint
  35. 35. Load Balancing – intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LBLB LB LB per container host • Service clients use an LB local to their host • E.g., Kube-proxy, Swarm, Mesos - MinuteMan
  36. 36. Ingress vs Intra-cluster differences ¨ Performance ¨ Count ¨ Frequency of reconfiguration ¨ L4 (intra-cluster) vs L7 (ingress) ¨ Form factor
  37. 37. Reconfiguration of Load Balancer α1 β1 HostH1 β2 α2 HostH2 α3 α4 HostH3 Ingress LB α5 α6 HostH4 ClusterManager LB Controller Cluster API Container Events Reconfigur ation Container State Container State Query
  38. 38. Nitrox – Configure Citrix NetScaler for Cluster Managers ¨ github.com/chiradeep/nitrox ¨ Apache license ¨ Support for integrating NetScaler with ¤ Kubernetes ¤ Swarm ¤ Consul ¤ Marathon
  39. 39. Microservices LB patterns ¨ Client-side LB ¨ Zero-downtime deploys ¨ Resilience patterns ¨ Visibility / Insights
  40. 40. Client-side LB ¤ Embedded into calling application n e.g., Netflix Ribbon (with Eureka) ¤ Run as side-cars (alongside each application or one- per-host) Eureka M M M M M M
  41. 41. Client-side LB ¤ Run as side-cars, or one-per-host Service Discovery M M M M M M Clients Service LB LB
  42. 42. Client-side LB Examples ¨ Netflix Prana - Hystrix as side car (deprecated) ¨ Linkerd from Buoyant.io (based on Twitter Finagle) ¨ AirBnB Smartstack/Synapse (uses HAProxy) ¨ Uber Hyperbahn ¤ Like Finagle, switched from HAProxy ¨ Kube Proxy ¤ Initially user-space, now based on IPTables ¨ Docker Swarm LB ¤ Based on IPVS ¨ Mesos LB ¤ Based on IPTables
  43. 43. Zero-Downtime Deploys ¨ Canary deploys ¨ Traffic can be selected based on source (country), headers M M M M Version N M M M M M Version N+1 10%90% CD Pipeline
  44. 44. Zero-Downtime Deploys ¨ Blue-green deploys M M M M Version N M M M M M Version N+1 100% CD Pipeline
  45. 45. Zero downtime deploys with NetScaler HostH1 HostH2 HostH3 Ingress NetScaler HostH4 ContainerClusterCluster API Container Events Reconfig uration Controller NetScaler CPXJenkins
  46. 46. Advanced Resilience – Circuit Breaker ¨ Circuit Breaker is a pattern that prevents cascading failures due to unresponsive downstream services ¨ A load balancer can be configured to ignore a slow backend Credit: http://martinfowler.com/bliki/CircuitBreaker.html
  47. 47. Visibility ¨ Compliance ¨ Topology ¨ Analytics M M M M M M M M M M M Analytics
  48. 48. Topology and Compliance Policy Violation
  49. 49. Wrap-up ¨ Load balancing is different for containers / microservices ¨ Integration with container cluster managers is needed ¨ Consider using the same LB technology for ingress and intra-cluster ¨ Emerging patterns solidify the importance of the load balancer ¨ LB in the wire brings ¤ Simplicity ¤ Resilience ¤ Future proofing
  50. 50. https://www.microloadbalancer.com

    Be the first to comment

    Login to see the comments

Presented at API World 2016, San Jose, CA

Views

Total views

545

On Slideshare

0

From embeds

0

Number of embeds

23

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×