Apache CloudStack 4.3 adds support for clouds built using Microsoft Hyper-V, in addition to supporting VMware vSphere, Citrix XenServer, KVM, Oracle VM, Linux Containers and bare metal options. This deck covers the decision points impacting the design of CloudStack 4.3 clouds, and their relationship with hypervisor choices. Presented at Build a Cloud Day co-located with SCaLE 12x in February 2014.

1. Hypervisor Selection in CloudStack 4.3
Understanding the choices available
Build a Cloud Day – SCALE 12x
Tim Mackey – XenServer Community Manager and Evangelist

2. Building a successful cloud
What are we trying to accomplish?

3. Service Offerings
• Clearly define what you want to offer
ᵒ What types of applications
ᵒ Who has access, and who owns them
ᵒ What type of access
• Define how templates need to be managed
ᵒ Operating system support
ᵒ Patching requirements
• Define expectations around compliance and availability
ᵒ Who owns backup and monitoring
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

4. Define Tenancy Requirements
• Department data local to department
ᵒ Where is the application data stored
• Data and service isolation
ᵒ VM migration and host HA
ᵒ Network services
• Encryption of PII/PCI
ᵒ Where do keys live when data location unknown
ᵒ Need encryption designed for the cloud
• Showback to stakeholders
ᵒ More than just usage, compliance and audits
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

5. Virtualization Infrastructure
• Hypervisor defined by service offerings
ᵒ
ᵒ
ᵒ
ᵒ
Don’t select hypervisor based on “standards”
Understand true costs of virtualization
Multiple hypervisors are “OK”
Bare metal can be a hypervisor
• To “Pool” resources or not
ᵒ Is there a real requirement for pooled resources
ᵒ Can the cloud management solution do better?
ᵒ Real cost of shared storage
• Primary storage defined by hypervisor
• Template storage defined by solution
ᵒ Typically low cost options like NFS
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

6. The primary choices ….

7. XenServer
Guest
Guest
Driver front
Driver front
Standard Linux Distribution (dom0)
xapi
patches
Driver back
drivers
qemu
Xen Project Hypervisor
Compute
Networking
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Storage

8. KVM (Linux + KVM only)
Guest
Guest
Virtual driver
Virtual driver
libvirt
Standard Linux Distribution
KVM Module
agent
Compute
virtio
drivers
Networking
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
qemu
Storage

9. vSphere 5.1 – Managed by vCenter
Guest
Guest
Virtual driver
Virtual driver
Service
Console
vCenter
vmkernel
Task
Scheduler
vNIC
vSCSI
vmklinux
Compute
Networking
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
drivers
Storage

10. Linux Containers
Namespace
Container
Namespace
Container
Cgroup
Cgroup
libvirt
Standard Linux Distribution
Namesspaces
Cgroups
Compute
Networking
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM Module
agent
Storage

11. Microsoft Hyper-V
Guest
Guest
Devices
Devices
VMBus
VMBus
Standard Windows Server (parent partition)
WMI
VMBus
Virtual SP
drivers
Hyper-V Hypervisor
Compute
Networking
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Storage

12. Defining the network

13. Flat Network – Basic Layer 3 Network
Option
XenServer
vSphere
KVM
LXC
Hyper-V
Security Groups
Yes- bridge
No
Yes
Yes
Yes
IPv6
No
No
Yes
Yes
No
Multiple IPs per NIC Yes
Yes
Yes
Yes
Yes
Nicira NVP
Yes
No
Yes
No
No
BigSwitch VNS
Yes
No
Yes
No
No
Public Network
65.11.0.0/16
Security Group 1
65.11.1.2
65.11.1.3
65.11.1.4
65.11.1.5
DHCP,
DNS
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack
Virtual Router
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
Security Group 2

14. VLANs for Private Cloud
Option
XenServer vSphere
KVM
LXC
Max VLANs
800
254
1024
1024 4094
IPv6
No
No
Yes
Yes
No
Multiple IPs
per NIC
Nicira NVP
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
No
Hyper-V
No
BigSwitch VNS
Yes
No
Yes
No
No
No
Yes
No
Public
Network/Internet
Public IP
65.37.14.1
No
MidoKura
Guest Virtual Network 10.0.0.0/8
VLAN 100
No
VPC
Yes
Yes
Yes
No
Yes
NetScaler
Yes
Yes
Yes
No
Yes
F5 BigIP
Yes
Yes
Yes
No
Yes
Juniper SRX
No
Yes
Yes
No
Yes
Cisco VNMC
No
Yes
No
No
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack
Virtual
Router
DHCP, DNS
NAT
Load Balancing
VPN
Gateway
10.1.1.1
10.1.1.1
10.1.1.3
10.1.1.4
10.1.1.5
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4

15. Beyond the VLAN – Network Virtualization
Option
XenServer
vSphere
KVM
LXC
Hyper-V
OVS GRE tunnels
Yes
No
No
No
No
Nicira STT tunnel
Yes
Yes
Yes
No
No
MidoNet
No
No
Yes
No
No
VXLAN
No
Yes
Yes
No
No
NVGRE
No
No
No
No
No
Nexus 1000v
No
Yes
No
No
No
Juniper Contrail
Yes
No
No
No
No
Palo Alto
Yes
Yes
Yes
No
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

16. Virtual Private Cloud and nTier Applications
Feature
PVLAN
XenServer
Yes - ovs
vSphere
Yes
KVM
ovs
LXC
No
DC2
Hyper-V
No
DC1
DC3
VLAN 1
DC4
Web
S2S VPN
Router
VLAN 2
App
Private
GW
VLAN 3
DC5
DC6
DB
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

17. Delivering specific network services
• KVM
ᵒ IPv6
ᵒ Security groups
ᵒ Large quantity of VLANs
• vSphere
ᵒ VXLAN required vSphere Enterprise Plus
ᵒ Cisco Nexus 1000v and ASA 1000v require vSphere Enterprise Plus
• XenServer
ᵒ Security groups
ᵒ Large quantity of VLANs
ᵒ Juniper Contrail
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

18. Instances need a home …
Storage, Storage and more Storage

19. Primary Storage Options
Feature
Local storage
NFS
SMB
Single path iSCSI
Multipath iSCSI
Direct array
Shared Mount
Template format
SolidFire Plugin
NetApp Plugin
Zone wide
Ceph RBD
Clustered LVM
XenServer
Yes
Yes
No
Yes
PreSetup
No
No
VHD
Yes
Yes
No
No
No
vSphere
Yes
Yes
No
Yes
No
VAAI
No
OVA
Yes
Yes
Yes
No
No
KVM
Yes
Yes
No
Yes
No
No
Yes
QCOW2
Yes
Yes
Yes
Yes
Yes
LXC
Yes
Yes
No
No
No
No
Yes
TAR
No
No
No
No
No
Hyper-V
Yes
No
SMB3
No
No
No
No
VHD
No
No
No
No
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Host
Host
Primary Storage
Cluster

20. Secondary Storage Options
Option
XenServer
vSphere
KVM
LXC
Hyper-V
NFS
Yes
Yes
Yes
Yes
No
Swift(1)
Yes
Yes
Yes
Yes
No
S3 compatible (2)
Yes
Yes
Yes
Yes
No
SMB
No
No
No
No
Yes
Host
(1) Requires NFS staging area
(2) Can be region wide, but must not have NFS secondary storage in zone
Host
Primary Storage
Cluster
Pod
Secondary Storage
Zone
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

21. Core virtualization capabilities
The limits and features which matter

22. CloudStack Features
Feature
XenServer
vSphere
KVM
LXC
Hyper-V
Disk IO Statistics
Yes
No
Yes
No
Yes
Memory Overcommit
Yes (4x)
Yes
No
No
No
Dedicated resources
Yes
Not with HA/DRS
Yes
No
Yes
Disk IO throttling
No
No
Yes
Yes
No
Disk snapshot (running)
Yes
Yes
No
No
No
Disk snapshot (pluggable)
Partial
Partial
No
No
No
Disk snapshot (Stopped)
Yes
Yes
Yes
No
Yes
Memory snapshot
Yes
Yes
Yes
No
No
Zone wide primary storage
No
Yes
Yes
Yes
No
Resize disk
Offline
Online Grow
Online
No
No
High availability
CloudStack
Native
CloudStack
No
CloudStack
CPU sockets
6.2 and higher
Yes
Yes
Host count
Yes
Affinity groups
Yes
Yes
Yes
No
Yes
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

23. XenServer 6.2
Feature
Source code model
Open Source (GPLv2)
Maximum VM Density
650 (Linux)
CloudStack VM Density
500
CloudStack integration
Direct XAPI calls
Maximum native cluster size
16
Maximum pRAM
1 TB
Largest VM
16vCPU/128GB
Windows Operating System
All Windows supported by Microsoft
Linux Operating Systems
RHEL, CentOS, Debian, Ubuntu, SLES, OEL
Advanced features supported
ovs, Storage XenMotion, DMC
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

24. vSphere 5.5 ( 5.5 support added in ACS 4.3)
Feature
Source code model
Proprietary
Maximum VM Density
512
CloudStack VM Density
128
CloudStack integration
vCenter
Maximum native cluster size
32
Maximum pRAM
4 TB
Largest VM
64 vCPU/1TB
Windows Operating Systems
DOS, All Windows Server/Client
Linux Operating Systems
Most
Advanced features supported
HA, DRS, vDS, Storage vMotion
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

25. KVM (RHEL/CentOS 6.5 and Ubuntu 12.04)
Feature
Source code model
Open Source (GPLv2)
Maximum VM Density
10 times the number of pCores
CloudStack VM Density
50
CloudStack integration
CloudStack Agent (libvirt)
Maximum native cluster size
No native cluster support
Maximum pRAM
2 TB
Largest VM
160 vCPU/2TB
Windows Operating Systems
Windows XP and higher
Linux Operating Systems
Varies
Advanced features supported
None
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

26. Linux Containers
Feature
Source code model
Open Source (GPLv2)
Maximum container Density
6000 (theoretical)
CloudStack container Density
50
CloudStack integration
CloudStack Agent (libvirt), requires KVM for SVMs
Maximum native cluster size
N/A
Maximum pRAM
2 TB
Largest container
2TB
Windows Operating Systems
N/A
Linux Operating Systems
Kernel compatible distros
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

27. Microsoft Hyper-V
Feature
Source code model
Proprietary
Maximum VM Density
1024
CloudStack VM Density
1024
CloudStack integration
CloudStack Agent (C# calling WMI)
Maximum native cluster Size
64
Maximum pRAM
4 TB
Largest VM
64 vCPU/1TB
Windows Operating Systems
All Windows supported by Microsoft
Linux Operating Systems
RHEL, CentOS, Debian, Ubuntu, SLES, OEL
Advanced features supported
None
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

28. Picking the “best one”
When to use which hypervisor…

29. KVM
• Primary value proposition:
ᵒ Low cost with available vendor support
ᵒ Familiar administration model
ᵒ Broad CloudStack feature set with active development
• Cloud use cases:
ᵒ
ᵒ
ᵒ
ᵒ
Linux centric workloads
Dev/test clouds
Web hosting
Tenant density which dictates SDN options
• Weaknesses:
ᵒ Requires use of an installed CloudStack libvirt agent
ᵒ Limited native storage options
ᵒ No use of advanced native features
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

30. Linux Containers
• Primary value proposition:
ᵒ Low cost with available vendor support
ᵒ Familiar administration model
• Cloud use cases:
ᵒ Dev/test clouds
ᵒ Web application hosting
• Weaknesses:
ᵒ
ᵒ
ᵒ
ᵒ
Requires use of an installed CloudStack libvirt agent
Requires KVM for system VMs
No use of advanced native features
First introduced in CloudStack 4.2
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

31. Microsoft Hyper-V
• Primary value proposition:
ᵒ Unlimited Windows Server VM licenses
ᵒ Familiar Windows management paradigm
• Cloud use cases:
ᵒ
ᵒ
ᵒ
ᵒ
Windows and Linux workloads
Dev/test clouds
.Net application web hosting
Desktop as a Service clouds
• Weaknesses:
ᵒ Minimal use of advanced native features
ᵒ First introduced with CloudStack 4.3
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

32. vSphere
• Primary value proposition:
ᵒ
ᵒ
ᵒ
ᵒ
ᵒ
Broad application and operating system support
Readily available pool of vSphere administration talent
Large eco-system of vendor partners
Many CloudStack features are native implementations
Direct feature integration via vCenter
• Cloud use cases:
ᵒ Private enterprise clouds
ᵒ Dev/test clouds
• Weaknesses:
ᵒ vSphere up-front license and ongoing support costs
ᵒ vCenter integration requires redundant designs
ᵒ Single data center per zone model
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

33. XenServer
• Primary value proposition:
ᵒ
ᵒ
ᵒ
ᵒ
Low cost with available vendor support
Broad CloudStack feature set with active development
Large CloudStack install base
Direct integration via XAPI toolstack
• Cloud use cases:
ᵒ
ᵒ
ᵒ
ᵒ
ᵒ
Linux centric workloads
Dev/test clouds
Web hosting
Desktop as a Service clouds
Large VM density and secure tenant isolation
• Weaknesses:
ᵒ Minimal use of advanced native features
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

34. What About Multiple Hypervisor Support?
• Networking
ᵒ Ensure network labels match
ᵒ Topology is intersect of chosen hypervisors
• Storage
ᵒ For system VMs to specific hypervisor type
ᵒ Zone with primary storage limited
• Operations
ᵒ
ᵒ
ᵒ
ᵒ
vSphere Datacenter can not span zones
Hyper-V may not be mixed with other hypervisors
HA won’t migrate between hypervisors
Capacity planning at the cluster/pod level more difficult
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

35. Tying it all Together
1. Define success criteria
2. Select a topology which works
3. Decide on storage options
4. Define supported configurations
5. Select preferred hypervisor(s)
6. Validate matrix
7. Build your Cloud
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy

36. Work better. Live better.