CloudStack Networking Deepdive CCCEU13
•
11 likes•3,023 views
Networking in CloudStack is full-featured, full of bells and whistles and by necessity complicated. This session will take cloud operators through the ins-and-outs of CloudStack Networking. Attendees will learn the motivations behind how CloudStack networking is architected, solutions to common networking requirements, gotchas, troubleshooting CloudStack networking and finally some future directions for theses features. It is assumed that the audience will have some experience administering CloudStack clouds.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to CloudStack Networking Deepdive CCCEU13
Similar to CloudStack Networking Deepdive CCCEU13 (20)
More from Chiradeep Vittal
More from Chiradeep Vittal (10)
Recently uploaded
Recently uploaded (20)
CloudStack Networking Deepdive CCCEU13
- 1. CloudStack Networking Deep Dive @chiradeep CloudStack Collabora9on Conference Amsterdam Nov 22, 2013
- 2. Overview • • • • • 10,000 foot overview of CloudStack Networking SoDware architecture Complex use cases Performance issues Focus on – Advanced zone – VLAN isola9on
- 3. Virtual Networking • The illusion of isolated networks on top of shared physical infrastructure • Requires orchestra9on of – Hypervisors – SoDware switches • Services are provided in virtual contexts – E.g., LB, firewalls
- 4. Virtual Network Services • Provide L2-‐L7 network services that applica9ons expect: – Load balancing, firewall, IDS, VPN, NAT, etc. • Services are inserted in the virtual network topology – usually in the path to the public network • Services are on-‐demand (api-‐driven), scalable, elas9c
- 5. Virtual Network Appliances Network services are oDen provided by virtual appliances. These are either commercial appliances in the virtual form factor or Linux-‐ based networking appliances Virtual Router! Public Network Nic! Control Network Nic! Virtual Network Nic!
- 6. Network Services Network Services • L2 connec9vity • IPAM • DNS • Rou9ng • ACL • Firewall • NAT • VPN • LB • IDS • IPS
- 7. Network Services Network Services • L2 connec9vity • IPAM • DNS • Rou9ng • ACL • Firewall • NAT • VPN • LB • IDS • IPS Service Providers! ü Virtual appliances! ü Hardware firewalls! ü LB appliances! ü SDN controllers! ü VRF! ü Hypervisor!
- 8. Network Services Network Services • L2 connec9vity • IPAM • DNS • Rou9ng • ACL • Firewall • NAT • VPN • LB • IDS • IPS Service Providers! ü Virtual appliances! ü Hardware firewalls! ü LB appliances! ü SDN controllers! ü VRF! ü Hypervisor! Network Isola1on • No isola9on • VLAN isola9on • Overlays • L3 isola9on
- 9. Network Offerings • Cloud users are not exposed to the nature of the service provider • Cloud operator designs a service catalog and offers them to end users. – Gold = {LB + FW, using virtual appliances} – Platinum = {LB + FW + VPN, using hardware appliances} – Silver = {FW using virtual appliances, 10Mbps}
- 10. Mul9-‐9er virtual networking Internet! IPSec VPN! ! Customer! Premises! VR! Loadbalancer (HW or Virtual) Private Gateway! Web VM 3! VLAN 398 VLAN 101 Web VM 2! App VM 2! VLAN 2724 App VM 1! Web VM 1! DB VM 1! Network Services! • IPAM! • DNS! • LB [intra]! • S-2-S VPN! • Static Routes! • ACLs! • NAT, PF! • FW [ingress & egress]!
- 11. Virtual networking with overlays Internet! IPSec VPN! ! Customer! Premises! VR + vSwitches! App VM 1! Web VM 2! Web VM 3! GRE KEY 398 GRE KEY 101 Web VM 1! App VM 2! GRE KEY 2724 Private Gateway! Loadbalancer (Virtual) DB VM 1! Network Services! • IPAM! • DNS! • LB [intra]! • S-2-S VPN! • Static Routes! • ACLs! • NAT, PF! • FW [ingress & egress]!
- 13. CloudStack Architecture 5 4 1 API API API Plugin Framew ork 2 Orchestra9on Engine Hyperviso Hyperviso r Plugins r Plugins 6 8 3 7 Network Network Plugins Plugins Allocator Storage Plugins Plugins 9 Hypervisor Hypervisor Resource Resource Network Network Resource Resource Storage Storage Resource Resource Allocator Allocator Plugins Plugins Physical Resources ! Orchestration steps usually executed in sequence!
- 14. Plugin interac9on 1 API API API Async Job Mgr 2 Plugin Frame 4 work Orchestra9on Engine Idempotent 5 Network Network Plugins Plugins 6 Desired State 3 CloudSt ack DB Idempotent 7 Network Network Resource Resource 8 Desired State Opera9onal State Desired State Plugin should not update CloudStack objects
- 15. Plugin Interac9on Details • Resource calls are expected to be idempotent – It is the job of the plugin to ensure this – apply(apply(config)) == apply(config) • Plugins should not update CloudStack resources – networks, rules, ip addresses, etc. – This is the desired state as expressed by the API • Plugins can have their own tables inside the CloudStack DB • Usually, there is no retry from the CloudStack orchestra9on engine upon failure – API reports failure. – Security groups mechanism will con9nuously retry (eventual consistency) – Refactor proposed to provide eventual consistency for all network plugins
- 16. USE CASES
- 17. Isolated vs. VPC • VPC should be the default choice – Isolated = VPC with a single 9er • Differences – Isolated: Remote Access VPN, Firewall Rules on Public IP – VPC: Site-‐to-‐Site IPSec VPN, ACL • RA VPN coming very soon • FW on public IP may come later – Use ACL for equivalent func9on • Use Isolated when firewall is – SRX – Cisco vASA1000
- 18. Real-‐life Use cases • • • • • • • • • Monitoring-‐as-‐a-‐Service Managed Applica9ons Cloud burs9ng Mix bare-‐metal/virtual workloads Load balance applica9on 9er MPLS VPN Mul9-‐zone failover Mul9ple IP usecases Metering with hardware appliances
- 19. Monitoring / Backup Service • Shared VLANs for monitoring/ backup • Mul9-‐tenancy enforced by using PVLAN (Xen/VMW/KVM) ! VR! App VM 2! VLAN 2724 Web VM 3! VLAN 398 VLAN 101 Web VM 2! DB VM 1! Backup PVLAN (shared)! App VM 1! Web VM 1! Cloud Operator Resources (aaS) Monitoring Server Backup Server Patching PVLAN (shared)! Patching Server Monitoring PVLAN (shared)! Tenant VPC
- 20. Monitoring Using Private Gateway Monitoring Server Private Gateway ! VR! Backup Server NAT Web VM 1! Web VM 2! App VM 2! Patching Server App VM 1! Web VM 3! Tenant VPC DB VM 1! Cloud Operator Resources • -‐ Addi9onal traffic through VR • -‐ Performance impact on VR • + Usable for SDN topologies
- 21. Managed Applica9ons Shared App Manager (not CloudStack Managed) VR Hosted App (CRM/VDI/ CMS/etc) for Tenant A VM VM VM VM VM NAT VM VR Hosted App for Tenant B Cloud Operator Resources VM VM VM VM VM NAT VM Public or Private GW • Mul9ple tenants share the applica9on manager/ orchestrator • Tenant will get charged if using public route.
- 22. S3 / Object Store VR VPC for Tenant A VM VM VM VM VM VM • Tenant will get charged for network bandwidth between object store and compute even if object store is co-‐located VR VPC for Tenant B VM VM VM VM VM VM Public Network
- 23. MPLS VPN Tenant A DC Private GW VR WAN Tenant A VM VM VM VM VM VM Tenant B DC VR Tenant B Private GW VM VM VM VM VM VM • S9tching VLAN to MPLS label is done outside CloudStack • Alterna9vely: use solu9on like Contrail SDN • No VLANs, use overlays • VR func9on replaced by logical distributed router
- 24. Cloud Burs9ng RFC 1918 Public IP Range reserved for Tenant A VR Internet VM VM VM VM VM VR Tenant B VM VM VM VM VM VM Tenant A DC Public VLAN 1 VM Public VLAN 2 Tenant A • Clients in Tenant DC need to use services inside VPC in the cloud • Tenant DC is already networked with Cloud (e.g., with MPLS VPN) • Public VLAN can be reserved for a tenant (use RFC1918 address range if needed). • Cloud operator can choose not to bill for ‘public’ data traffic
- 25. Internal Load Balancer ! VR! Web VM 1! Web VM 2! Web VM 3! LB VM 1! • LB VM lifecycle is managed by CloudStack App 1 VM 1! App 1 VM 2! App 1 VM 3! LB VM 2! App 2 VM 1! App 2 VM 2! DB VM 1!
- 26. Interoperate with legacy appliances / Baremetal DB Use reserved IP range VM HW appliances not managed by CloudStack VPN VM VM VM CS VM but not configured by CS Shared VLAN 200 Can be RFC1918 or Public IP Not CloudStack-‐ Managed Baremetal DB Shared VLAN 100 • Managed hos9ng primarily • Baremetal advanced zone support forthcoming
- 27. Conundrum: Inter-‐AZ private traffic Tenant A Tenant A Tenant B Zone 1 Tenant B High speed Private Interconnect Zone 2
- 28. Inter-‐AZ private traffic • Solu9on 1: Use VR-‐VR IPSec VPN – Coming in 4.3 – Cons: • ipsec overhead, • tricky usage accoun9ng, • Uses public internet • Solu9on 2: overlay – GRE tunnel between VRs – Not yet a proposal, just a possibility • Solu9on 3: SDN overlay – Stretch overlay subnets between DC – Cons: • Needs change to CloudStack model (network belongs to zone) • SDN controllers inherently are single-‐DC
- 30. VR performance • FAQ: what is the performance of the VR? – # of concurrent connec9ons? – # of connec9ons per second? – Throughput? – Latency? • Answer: – “It depends”
- 31. Factors affec9ng performance • Physical network – Public network capacity (ports/buffers/etc.) – Top of rack switch capacity • Hypervisor – CPU model (L1/L2 cache, clock speed) – NICs (1 GigE/10GigE/link aggrega9on mode/Jumbo Frames) – RAM and CPU allocated to dom0 – Bridge vs OVS • Virtual Router specs – CPU speed – RAM – (4.2.1) 32-‐bit vs 64-‐bit
- 32. Topology for tes9ng Hypervisor H1 VR Web VM 2 Hypervisor H2 Tool Host Web VM 1 Web VM 3 Web VM 4 Hypervisor H3 Public Network Guest Network
- 33. Do your own tes9ng • Basic Topology – Intra hypervisor, guest VM -‐> guest VM • VM3 -‐> VM4 – Between hypervisors, guest VM -‐> guest VM on same VLAN • VM1 -‐> VM3 • Routed topology – VR between traffic source and des9na9on • VR on same hypervisor as des9na9on – Tools Host -‐>VR -‐> VM2 (port forward) • VR on different hypervisor as des9na9on – Tools Host -‐> VM1 (port forward) • LB test – Tools Host -‐> {VM1, VM2, VM3, VM4}
- 34. VLAN Issues • En9re VLAN range trunked to all hypervisors – Unlimited broadcast domain – Per-‐port VLAN limits in certain switches • No good solu9ons: – VTP (Vlan Pruning) – Arista VMTracer (vSphere only) • Overlays (aka SDN) is probably the way to go