Using Next Generation Storage to Improve Performance, Scalability & Efficiency in your CloudStack Infrastructure

1. Using Next Generation Storage to
Improve Performance, Scalability &
Efficiency in your CloudStack
Infrastructure
Giles Sirett
CEO
Giles.sirett@shapeblue.com
Twitter: @ShapeBlue

2. Agenda








Key Design Considerations for a CloudStack IaaS Cloud
Primary & Secondary Storage
Scaling Storage
The IOPS Dilemma
Real World Storage Scenarios
Solidfire’s Next Generation Storage Platform
Demo of Solidfire / CloudStack Integration
Q&A
@ShapeBlue

3. Todays Speakers
Giles Sirett
CEO
ShapeBlue
Mike Tutkowski
Senior CloudStack Developer
SolidFire
Geoff Higginbottom
CTO
ShapeBlue
McClain Buggle
Strategic Alliance Manager
SolidFire
@ShapeBlue

4. Questions


Please use the GTM question feature.
Don’t wait until the end !
@ShapeBlue

5. About ShapeBlue
“ShapeBlue are expert builders of public & private
clouds. They are the leading global independent
CloudStack / CloudPlatform integrator &
consultancy”
@ShapeBlue

6. @ShapeBlue

7. About ShapeBlue
@ShapeBlue

8. How to build an IaaS cloud
PaaS
Billing
Developer
Tooling
Multi-cloud
Management
eCommerce Platform
Ecommerce
Management
orchestration API
CloudStack API
CMP - Orchestration layer
Apache CloudStack
Choice of Hypervisor
Hypervisor
KVM, VMware, XenServer, Hyper-V
Networking
Commodity
Compute
Compute
Storage
@ShapeBlue

9. CloudStack Architecture
Region
Zone
POD
Secondary
Storage
Cluster
Host(s)
Primary
Storage
@ShapeBlue

10. Secondary Storage







Configured at Zone level
Has to be accessible by Management Servers, Hosts and SSVMs
Stores all Templates, ISOs and Snapshots
Has to be NFS
Swift and S3 can be used to replicate data between Zones
Zones can have more than one Secondary Storage Volume
Public Templates & ISOs are copied to every store within a Zone
@ShapeBlue

11. Scaling Secondary Storage

Capacity determined by



Number of Public ISOs
Number of Public Templates
Data Consumed by Users




Volume Snapshots
Private ISOs
Private Templates
Uploaded Volumes
@ShapeBlue

12. Primary Storage



Configured at Cluster or Zone level
Stores all Disk Volumes and VM Snapshots for VMs
Options are:





NFS
iSCSI
FC
Local Storage
High Performance is critical
@ShapeBlue

13. Geoff Higginbottom
CTO ShapeBlue
www.shapeblue.com
CloudStack Collaboration Conference 2012

14. Considerations for Scaling Primary Storage

Workload is a key factor



Optimal LUN Size for iSCSI ~ 500GB



Capacity required for each VM
Disk performance required for each VM
20 VMs per iSCSI LUN for optimal performance
Significant admin overhead to create and manage
Admins tend to create large LUNs for ease of management


Performance Suffers
Scale-Out is a manual process
@ShapeBlue

15. Cloud Scale Requires Automation


Cloud scale requires automation of all components
CloudStack can orchestrate






Hypervisors
Networks (SDN)
Network Appliances – F5, LB, Netscaler etc
Virtual Machines
Storage is a very significant piece of the infrastructure
Choose a storage vendor who integrates with CloudStack
@ShapeBlue

16. Scaling Primary Storage - Example

4 XenServer Clusters, each with 4 Hosts



28 usable cores (after Hypervisor overhead)
764GB usable RAM (after Hypervisor overhead)
Mapping of 6vCPU to 1 core
@ShapeBlue

17. Scaling Primary Storage - Example


Average VM Specification is 1vCPU / 4GB of RAM
Based on total host capacity

2528 Guest VMs



75,840 IOPs (30 IOPS per VM)
50.5 TB of Storage (20GB per VM)
1264 VRs


12640 (10 IOPS per VR)
2.5 TB of Storage (2 GB per VM)
@ShapeBlue

18. Storage Array IOPS vs Capacity - Example

What we need



What can a traditional storage array provide (real world figures)



53 TB capacity
88,480 IOPS
50TB Capacity
9,500 IOPS
Therefore we need 10 Storage Arrays


500TB Capacity
95,000 IOPS
10x the required capacity to
achieve required IOPS
@ShapeBlue

19. Extract from CloudStack Design Document
@ShapeBlue

20. Capacity Overhead – Real World Example

Requirements:



12TB of Storage
19,500 IOPS
Solution:


100TB Storage Capacity
19,000 IOPS
x6 Storage Arrays = 528TB of wasted capacity
@ShapeBlue

21. Extract from CloudStack Design Document
@ShapeBlue

22. Other Factors Affecting Capacity

Linked Clones



Each VM within a LUN/Volume is linked to a ‘Template’ image
Template may have 20GB allocated but only 2GB on disk
Linked VMs only contain the differences – starts at 200MB

Thin Provisioning
Deduplication

Actual storage space required is much lower than you think!

@ShapeBlue

23. IOPS vs Capacity Problem




Total IOPS per Storage Array – not per VM
Traditional Multi-Tiered Storage has its limits
Forced to add excess capacity to achieve IOPS
Deduplication actually makes matters worse
@ShapeBlue

24. McClain Buggle
Strategic Alliance Manager
SolidFire

25. Key Primary Storage
Considerations
Guaranteed
Performance
Fine-grain performance
management with Guaranteed
Quality of Service (QoS)
Scale-out
Simultaneous scaling of both
capacity and performance
In-line
efficiency
In-line data reduction reduces data
footprint, requires less purchased capacity
High
Availability
RAID-less data protection maintains data
availability, security and performance
regardless of failure condition.
Management
Automation
Intuitive UI, CLI, and REST-based
API for ease of management
complete automation

26. Scale-Out Architecture
Linear Scalability
96TB
5, SF3010 Nodes
• 60 TB
• 250,000 IOPS
Starter 5 node
SF3010 configuration
Capacity
84TB
400,000 IOPS
350,000 IOPS
72TB
300,000 IOPS
60TB
250,000 IOPS
Performance
• New nodes are added as demand dictates
• Performance and capacity instantly available to all volumes
• Nodes added on the fly without down time

27. High Availability
•
Cluster wide RAID-less data protection,
SolidFire Helix™
•
Self-healing – Automatic and complete restoration of
redundancy after failure
•
Complete HA – No single points of failure
•
Non-disruptive upgrades
9
Maintains all QoS settings in the face of component
and/or system level failure
•
7
Non-controller based architecture
•
6
4
3
Failure Isolation – without performance impact
•
1
2
A
7
5
5
8
8
4
6
3
2
1
A
9

28. Guaranteed Performance
• Allocate: Storage performance
independent of capacity
• Manage: performance real-time without
impacting other volumes
• Guarantee: performance to every
volume with fine-grain QoS settings

29. Efficiency, In-line
•
Always-on, in-line Data Reduction
– Deduplication
– Compression
– Thin Provisioning
•
Executed across entire data store without
performance impact
•
Delivers drastic reduction in power, cooling,
and floor space

30. System Automation
• Integrated REST-based API
• Enables complete automation of any SolidFire
function
• Supports development of user-facing storage
controls
• Seamless integration into current
billing/charge-back systems and management
stacks
• API Integration Support from SolidFire

31. Avoid Sprawling Capacity to Gain Performance
FAS6250AE
FAS6250AE
96RU - 391 TB
117,000 IOPS
84RU - 328 TB – 97,500 IOPS
SF9010 - 5RU - 174 TB - 375,000 IOPS
76RU - 296 TB – 84,500 IOPS
Power = Average 220W/RU
5U = 1,100W
68RU - 224TB – 71,500 IOPS
6250AE - 44RU - 174 TB - 52,000 IOPS
Power = Average 1000W/RU
44RU = 44,000W

32. Demo
Mike Tutkowski
Senior CloudStack Developer
SolidFire

33. Questions
@ShapeBlue

34. Further Information or questions slide

Shapeblue.com/blog

Slides can be found at





www.slideshare.net/shapeblue
Giles.sirett@shapeblue.com
Geoff.higginbottom@shapeblue.com
@shapeblue
@cloudstackgurou
@ShapeBlue

35. Using Next Generation Storage to
Improve Performance, Scalability &
Efficiency in your CloudStack
Infrastructure
Giles Sirett
CEO
Giles.sirett@shapeblue.com
Twitter: @ShapeBlue