Apache CloudStack is a mature IAAS platform designed for scale and ease-of-use. However new cloud administrators typically struggle with networking in Apache CloudStack. Networking in CloudStack is full-featured, full of bells and whistles and by necessity complicated. This session will take the audience through the ins-and-outs of CloudStack Networking. Attendees will learn the motivations behind how CloudStack networking is architected, solutions to common networking requirements and future work. About Chiradeep Chiradeep Vittal is Distinguished Engineer in the Networking and Cloud Group at Citrix Systems. He is a maintainer in the Apache CloudStack project where he contributes to networking and storage parts of the Infrastructure-as-a-Service (IAAS) management system. He was a founding engineer at Cloud.com whose product CloudStack is now Apache CloudStack. CloudStack is deployed in more than 300 public and private clouds and powers some of the largest clouds in the world today.

1. Introduction to CloudStack
Networking
Silicon Valley CloudStack Meetup
9th October 2014

2. About me

3. Agenda
• Introduction to CloudStack
• Networking modes in CloudStack
• Virtual Networking
• Networking Internals
• Advanced Topics

4. Apache CloudStack
Apache CloudStack is a
• scalable,
• multi-tenant,
• open source,
• purpose-built,
• cloud orchestration platform for
• delivering turnkey Infrastructure-as-a-Service clouds

5. 300+
Large Scale
Production Clouds
In Deployment
Production sites with over
40,000+
Enterprise and
Education
Service Providers and
Servers Web
2.0
Telcos

6. How did Amazon build its cloud?
Amazon eCommerce Platform
AWS API (EC2, S3, …)
Amazon Orchestration Software
Open Source Xen Hypervisor
Commodity
Servers
Commodity
Storage
Networking

7. How can YOU build a CloudStack
cloud?
Amazon eCommerce Platform
Optional Portal
AWS API (EC2, S3, …)
CloudStack or AWS API
CloudStack Orchestration Software
Amazon Orchestration Software
Hypervisor
(XenServer/KVM/vSphere/Hyper-
Open Source Xen Hypervisor
V/LXC)
Networking Servers Storage

8. Image
Secondary Storage
End users
DC Edge
L3/L2 core
Zone Architecture
VM
Pod Pod Pod Pod
CloudStack
Pod
Access Sw
MySQL
Admin/User API
Hypervisor (Xen
/VMWare/KVM)
Primary Storage
NFS/ISCSI/FC
VM
VM
Image
Disk Disk

9. End users
Networking concerns in a cloud
Pod Pod Pod Pod
Pod
VM
VM
Disk Disk

10. Networking Concerns
• Network virtualization
– Multi-tenancy
• Network services for virtual networks and
machines
• Network automation
• Scalability

11. Networking Principles in Apache
CloudStack
• Flexibility
– Allow various combinations of technology for L2-L7 network services
– Allow different providers (vendors) for the same network service in a
Cloud POP
• Pluggability
– Plugins allow vendors to drop in vendor-specific configuration and
lifecycle management code
• Service scalability
– Scale out using virtual appliances when possible
– Scale up using hardware appliances if needed

12. Network Flexibility
Network Services
• L2 connectivity
• IPAM
• DNS
• Routing
• ACL
• Firewall
• NAT
• VPN
• LB
• IDS
• IPS
Network Isolation
• No isolation
• VLAN isolation
• Overlays
• L3 isolation
Service Providers
 Virtual
appliances
 Hardware
firewalls
 LB appliances
 SDN controllers
 IDS /IPS
appliances
 VRF
 Hypervisor

13. Networking Modes
• “Basic” mode
– L3 isolation
– Tenants share subnets
– VMs placed into security groups
• ACL governs communication between/within groups/outside
– No VLANs
– Excellent scaling (10s of thousands of hosts/VM)
– Limited network services
– Distributed network firewall using iptables on the hypervisor

14. Layer 3 cloud networking
…
DB
Security
Group
Web
Security
Group
DB
VM
Web
VM
… …
Web
VM
Web
VM
Web
VM
Web
VM
Web
VM
DB
VM
Ingress Rule: Allow VMs in Web Security Group access to VMs in DB Security Group on Port 3306

15. L3 isolation with distributed firewalls
Tenant
1 VM 1
10.1.0.2
Tenant
2 VM 1
10.1.0.3
Tenant
1 VM 2
10.1.0.4
Public
Internet
10.1.0.1
Public IP address
65.37.141.11
65.37.141.24
65.37.141.36
65.37.141.80
L3 Core
Load
Balancer
Pod 1 L2
Switch
Pod 2 L2 10.1.8.1 …
Switch
Pod 3 L2
Switch
10.1.16.1
…

16. L3 isolation with distributed firewalls
Tenant
1 VM 1
10.1.0.2
Tenant
2 VM 1
10.1.0.3
Tenant
1 VM 2
10.1.0.4
Tenant
1 VM 3
10.1.16.47
Tenant
1 VM 4
10.1.16.85
Public
Internet
10.1.0.1
Public IP address
65.37.141.11
65.37.141.24
65.37.141.36
65.37.141.80
L3 Core
Load
Balancer
Pod 1 L2
Switch
Pod 2 L2 10.1.8.1 …
Switch
Pod 3 L2
Switch
10.1.16.1
…

17. L3 isolation with distributed firewalls
Tenant
1 VM 1
10.1.0.2
Tenant
2 VM 1
10.1.0.3
Tenant
1 VM 2
10.1.0.4
Tenant
2 VM 2
10.1.16.12
Tenant
2 VM 3
10.1.16.21
Tenant
1 VM 3
10.1.16.47
Tenant
1 VM 4
10.1.16.85
Public
Internet
10.1.0.1
Public IP address
65.37.141.11
65.37.141.24
65.37.141.36
65.37.141.80
L3 Core
Load
Balancer
Pod 1 L2
Switch
Pod 2 L2 10.1.8.1 …
Switch
Pod 3 L2
Switch
10.1.16.1
…

18. 1 Firewall per Virtual Machine

19. VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
…
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
VM
…
VM
VM
A Million Firewalls?

20. Networking Mode: Advanced
• Network virtualization
– Networks can have the same subnet range
– Routing, ACL between networks
– Services provided at the edge
• NAT, Firewall, LB, VPN, etc

21. Virtual Network Appliances
Network services are often provided by virtual appliances.
These are either commercial appliances in the virtual form factor or Linux-based
networking appliances
Virtual Router
Public Network Nic Virtual Network Nic
Control Network Nic

22. Multi-tier virtual networking
VLAN 2724
DB
VM 1
Web
VM 1
Web
VM 2
Web
VM 3
VLAN 101
App
VM 1
App
VM 2
VLAN 398
VR
Internet
Customer
Premises
IPSec VPN
Private Gateway
Loadbalancer
(HW or
Virtual)
Network Services
• IPAM
• DNS
• LB [intra]
• S-2-S VPN
• Static Routes
• ACLs
• NAT, PF
• FW [ingress & egress]

23. Virtual networking with overlays
GRE KEY 2724
DB
VM 1
Web
VM 1
Web
VM 2
Web
VM 3
GRE KEY 101
VR + vSwitches
App
VM 1
App
VM 2
GRE KEY 398
Internet
Customer
Premises
IPSec VPN
Loadbalancer Private Gateway
(Virtual)
Network Services
• IPAM
• DNS
• LB [intra]
• S-2-S VPN
• Static Routes
• ACLs
• NAT, PF
• FW [ingress & egress]

24. Network Offerings
• Cloud users are not exposed to the nature of the service
provider
• Cloud operator designs a service catalog and offers them
to end users.
– Gold = {LB + FW, using virtual appliances}
– Platinum = {LB + FW + VPN, using hardware appliances}
– Silver = {FW using virtual appliances, 10Mbps}

25. Example: Network Service offering

26. CLOUDSTACK ARCHITECTURE

27. CloudStack Architecture
Plugin
Framew
ork
Orchestration Engine
Hyperviso
r Plugins
Hyperviso
r Plugins
Network
Plugins
Network
Plugins
Allocator
Plugins
Storage
Plugins
APIA
PI
API
Hypervisor
Resource
Hypervisor
Resource
Network
Resource
Network
Resource
Storage
Resource
Storage
Resource
Physical Resources
Allocator
Plugins
Allocator
Plugins
1
2
4
3
5
6
7
8
9
Orchestration steps usually executed in sequence

28. Plugin interaction
Orchestration
Engine
Plugin
Frame
work Network
Network
Plugins
Plugins
API
API
API
Network
Resource
Network
5
1 2 Resource
CloudSt
ack DB
Desired State
3
Desired State
4
Async
Job
Mgr
6
Operational State
Desired State
7
8
Idempotent
Idempotent
Plugin should not update
CloudStack objects

29. Plugin Interaction Details
• Resource calls are expected to be idempotent
• Plugins should not update CloudStack
resources
• Plugins can have their own tables inside the
CloudStack DB
• No automatic re-tries