Evaluation of Virtual Clusters Performance on a Cloud Computing Infrastructure
1. Evaluation of
Virtual
Clusters
Performance
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
Introduction
Hypothesis
Goals Evaluation of Virtual Clusters Performance on
Methods
a Cloud Computing Infrastructure
Results
Conclusions
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa
Universidade do Porto, Faculdade de Engenharia
November 4, 2011
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 1 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
2. Evaluation of
Virtual
Clusters
Performance
Outline
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito 1 Introduction
Vieira and
Jorge G.
Barbosa
2 Hypothesis
Introduction
Hypothesis
3 Goals
Goals
Methods
Results 4 Methods
Conclusions
5 Results
6 Conclusions
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 1 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
3. Evaluation of
Virtual
Clusters
Performance
User’s perspective
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
• Increase of computing needs
Introduction
Hypothesis
• Lack of flexibility
Goals
• SW requirements different from the available
Methods
• Permissions to the available platforms are restricted
Results
• Clusters exist for specific applications
Conclusions
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 2 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
4. Evaluation of
Virtual
Clusters
Performance
Provider’s perspective
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G. • High energy cost
Barbosa
Introduction • HW aging without proper use
Hypothesis
Goals
• SW is tightly coupled with HW leading to:
Methods • Difficult and lengthy migrations
Results
• Inflexibility
Conclusions
• Difficult substitution of failing components
• Difficulty in meeting users’ requirements
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 3 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
5. Evaluation of
Virtual
Clusters
Performance
Hypothesis
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
Introduction
Hypothesis Cloud Computing enables a better usage of the available
Goals hardware without seriously compromising performance.
Methods
Results
Conclusions
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 4 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
6. Evaluation of
Virtual
Clusters
Performance
Goals
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
• Explore the creation steps of an IaaS Cloud:
Introduction • In an academic context
Hypothesis • Using both specialized and non-specialized hardware
Goals • Using open source technologies
Methods
• Evaluate the losses introduced by virtualization
Results
Conclusions
• Analyze the benefits of the Cloud paradigm
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 5 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
7. Evaluation of
Virtual
Clusters
Performance
Resources characterization (1)
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
• More non-specialized HW
Introduction than specialized HW
Hypothesis • Specialized HW older than
Goals non-specialized HW
Methods
• Inferior performance
Results • Higher energy
Conclusions consumption Specialized: 256 Cores Non-Specialized: 3328 Cores
• Closer to failure
Figure: Computer resources
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 6 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
8. Evaluation of
Virtual
Clusters
Performance
Resources characterization (2)
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
Introduction
• High uptimes
Hypothesis • 70% > 40m
Goals
• 42% > 1h30
Methods • Low loads
Results
• Possibility of running jobs 10 < t <= 40: 30% 40 < t <= 90: 28%
Conclusions 90 < t <= 160: 23% 160 < t <= 250: 11%
during those periods 250 < t <= 360: 5% 360 < t <= 490: 3%
Figure: Non-specialized HW
uptimes
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 7 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
9. Evaluation of
Virtual
Clusters
Performance
Resources characterization (3)
on a Cloud
Computing
Infrastructure
• Tier 1 nodes:
Nuno
Cardoso, Tito • Hardware:
Vieira and
Jorge G.
• 2x Intel Xeon X3440
Barbosa (4 cores each)
• 24GB RAM
Introduction
• GbE network
Hypothesis
• Operating System:
Goals
Debian GNU/Linux 6
Methods
• Tiers 2 and 3 nodes:
Results
• Hardware:
Conclusions
• Intel Core 2 Quad
Q9300 (4 cores)
• 4GB RAM Figure: Multi-tier setup
• GbE network
• Operating System:
Ubuntu FEUP
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 8 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
10. Evaluation of
Virtual
Clusters
Performance
Technologies
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and Infrastructure Benchmarks
Jorge G.
Barbosa • Debian GNU/Linux 6 • Phoronix Test Suite 3.2
Introduction
• OpenNebula 2.0.1 • HPCC 1.4.1
Hypothesis
Goals
• KVM 0.12.5 • OpenMPI 1.4.2
Methods
Results • OpenVPN 2.1 • Condor 7.4.4
Conclusions
• OpenSSH 5.5 • Riak 0.14.1
• Cassandra 0.7.5
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 9 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
11. Evaluation of
Virtual
Clusters
Performance
Benchmark Summary
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G. Name Scope
Barbosa
Deployment Infrastructure
Introduction
Phoronix CPU/RAM, Disk IO
Hypothesis
HPCC CPU/RAM, Network, HPC
Goals
Methods
Condor Infrastructure, HTC
Results
Distributed DBs Infrastructure, Databases
Conclusions
Table: Benchmark Summary
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 10 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
12. Evaluation of
Virtual
Clusters
Performance
Deployment
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Setup
Jorge G.
Barbosa S1: 1 node → 1 VM
Introduction
S2, S4 and S6: 1 node at 1x, 2x and 4x capacity (8,16 and 32 VMs)
Hypothesis S3, S5 and S7: 2 nodes at 0.5x, 1x and 2x capacity (8,16 and 32 VMs)
Goals
VMs
Methods
Results
• 1 Virtual CPU
Conclusions • 1GB RAM
• 2GB Disk (image: 500MB)
• 4GB Swap
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 11 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
13. Evaluation of
Virtual
Clusters
Performance
Deployment
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Results
Vieira and
Jorge G.
• 1 VM → ∼ 1.8 sec
Barbosa
• Elastic capabilities
Introduction • Enables duplication of
Hypothesis VMs/Clusters
Request Transfer Boot
Goals • Transfer bottleneck: Avg Avg Avg
Methods S1 0.6 2.2 216.0
Distributed storage
S2 0.1 1.9 245.4
Results S3 0.1 1.3 228.5
• Processing power S4 0.1 1.1 466.7
Conclusions
S5 0.1 1.3 243.8
bottleneck: S6 0.1 1.5 717.4
More physical resources S7 0.1 1.0 418.2
Table: Benchmark results(sec)
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 12 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
14. Evaluation of
Virtual
Clusters
Performance
Benchmark Summary
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G. Name Scope
Barbosa
Deployment Infrastructure
Introduction
Phoronix CPU/RAM, Disk IO
Hypothesis
Goals
HPCC CPU/RAM, Network, HPC
Methods
Condor Infrastructure, HTC
Results Distributed DBs Infrastructure, Databases
Conclusions
Table: Benchmark Summary
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 13 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
15. Evaluation of
Virtual
Clusters
Performance
Phoronix Test Suite
on a Cloud
Computing
Infrastructure Setup Application Loss
Bullet Physics Engine 2.75 1.29%
Nuno
Cardoso, Tito • Virtualized vs. C-Ray 1.1 0.67%
Vieira and FFmpeg 0.6.1 5.39%
Jorge G. Non-Virtualized FLAC Audio Encoding 1.2.1 1.43%
Barbosa GraphicsMagick 1.3.12 2.33%
• Two Categories: John The Ripper 1.7.3.1 0.64%
LAME MP3 Encoding 3.98.2 4.46%
Introduction • CPU intensive OpenSSL 1.0.0a 0.55%
Smallpt 1.0 0.74%
Hypothesis • IO intensive
Goals
Results Table: CPU intensive
Methods
Results
• CPU virtualization → Low Application Loss
Dbench 4.0 1 -55.34%
Conclusions overhead (avg: 1.71%) Flexible IO Tester 1.21 73.12%
PostMark 1.51 -363.33%
• IO virtualization → High SQLite 3.7.3 -5.37%
7-Zip Compression 9.13 6.68%
overhead (avg w/o gains: 22.51%) GnuPG 1.4.10 1.26%
Unpacking linux-2.6.32 35.53%
• Needing further analysis: x264 2010-11-22 10.95%
Compile linux-2.6.32 4.90%
Cache
Table: IO intensive
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 14 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
16. Evaluation of
Virtual
Clusters
Performance
Benchmark Summary
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G. Name Scope
Barbosa
Deployment Infrastructure
Introduction
Phoronix CPU/RAM, Disk IO
Hypothesis
Goals
HPCC CPU/RAM, Network, HPC
Methods
Condor Infrastructure, HTC
Results Distributed DBs Infrastructure, Databases
Conclusions
Table: Benchmark Summary
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 15 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
17. Evaluation of
Virtual
Clusters
Performance
HPCC - Single Node
on a Cloud
Computing
Infrastructure Setup HPL
1
Nuno 0.8
Cardoso, Tito • 8 processes PTRANS
0.6
Latency
Vieira and
0.4
Jorge G.
Barbosa • 1 Computer (8 cores) 0.2
Random
Bandwidth
Access
Introduction
• Several “PxQ” and “N”’s
Hypothesis
Goals
• Virt. vs. Non-virt. FFTE DGEMM
Methods
Results
Results STREAM
VM Host
Conclusions
Spatial/Temporal Locality
Figure: Result comparison
High: Low losses
Temporal
(HPL: 2.2%) Low High
Low Random Access FFT
Spatial
PTRANS HPL
Low: High losses High
STREAM DGEMM
(RandomAccess: 35%)
Table: Data Locality
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 16 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
18. Evaluation of
Virtual
Clusters
Performance
HPCC - Multiple Nodes
on a Cloud
Computing
Infrastructure Setup HPL
1
Nuno 0.8
Cardoso, Tito • 16 processes (2x8) PTRANS
0.6
Latency
Vieira and
0.4
Jorge G.
Barbosa • 2 Computer (16 cores) 0.2
Random
Bandwidth
Access
Introduction
• Several “PxQ” and “N”’s
Hypothesis
Goals
• Virt. vs. Non-virt. FFTE DGEMM
Methods
Results
Results STREAM
VM Host
Conclusions
Huge impact on networking:
Figure: Result comparison
Latency: 563% increase
Temporal
Low High
Bandwidth: 70% decrease Spatial
Low Random Access FFT
High PTRANS HPL
STREAM DGEMM
Table: Data Locality
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 17 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
19. Evaluation of
Virtual
Clusters
Performance
Benchmark Summary
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G. Name Scope
Barbosa
Deployment Infrastructure
Introduction
Phoronix CPU/RAM, Disk IO
Hypothesis
Goals
HPCC CPU/RAM, Network, HPC
Methods
Condor Infrastructure, HTC
Results Distributed DBs Infrastructure, Databases
Conclusions
Table: Benchmark Summary
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 18 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
20. Evaluation of
Virtual
Clusters
Performance
Condor
on a Cloud
Computing
Infrastructure Setup
Nuno
Cardoso, Tito • S1: Baseline
Vieira and
Jorge G. 8 VMs → 4 (Tier 1) Computers
Barbosa
Introduction
• S2: Scalability/Resource
Hypothesis exploitation
Goals 8 + 15 VMs → 4 (Tier 1) + 15 (Tier 2) Computers
Methods
Results
• S3: Elasticity/Resource exploitation
Conclusions 15 VMs → 15 (Tier 3, 10a.m. until 19p.m.) Computers
Figure: Multi-tier setup
• Job: Subdivided into 10 minute
chunks, Embarrassingly Parallel
• VMs: 4 CPUs, 3GB RAM
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 19 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
21. Evaluation of
Virtual
Clusters
Performance
Condor
on a Cloud
Computing
Infrastructure
Nuno
Results
Cardoso, Tito
Vieira and • Execution times:
Jorge G.
Barbosa • 1 VM: 1 month (extrapolated)
• S1: 29 hours
Introduction
• S2: 10 hours
Hypothesis
• S3: 74 hours (extrapolated)
Goals
Methods • Nearly linear gain
Results
Conclusions • Possibility of using HW in both dedicated (S1 and S2) and
non-dedicated (S3) fashions
• Enables using unexploited (Tiers 2 and 3) HW to perform
tasks with no extra expense
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 20 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
22. Evaluation of
Virtual
Clusters
Performance
Benchmark Summary
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G. Name Scope
Barbosa
Deployment Infrastructure
Introduction
Phoronix CPU/RAM, Disk IO
Hypothesis
Goals
HPCC CPU/RAM, Network, HPC
Methods
Condor Infrastructure, HTC
Results Distributed DBs Infrastructure, Databases
Conclusions
Table: Benchmark Summary
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 21 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
23. Evaluation of
Virtual
Clusters
Performance
Distributed Databases
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Setup
Vieira and
Jorge G. • Baseline
Barbosa
6 nodes (2 CPUs, 4GB RAM), 1GB data per node
Introduction
Hypothesis
• S1: Elasticity
Goals Start from baseline; add 1 node; measure latencies
Methods
Results
• S2: Scalability
Conclusions Start from baseline; scale up to 12 nodes while measuring latencies
• Databases:
• Riak
• Cassandra
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 22 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
24. Evaluation of
Virtual
Clusters
Performance
Distributed Databases
on a Cloud
Computing
Infrastructure
Results 2000 ms
1500 ms
Nuno
Cardoso, Tito
• IO was a bottleneck
1000 ms
Vieira and
Jorge G. 500 ms
Barbosa • S1: The Cloud introduced
0 ms
elasticity in the cluster
0m
5m
10m
15m
20m
25m
30m
35m
40m
45m
50m
55m
60m
Introduction
Hypothesis
with a transient (15 min) Cassandra Riak
Goals overhead
Figure: S1 Result
Methods
Results
• S2: The cluster scaled 20 ms
19 ms
Conclusions while maintaining latency 18 ms
17 ms
constant (4 ms variation) 16 ms
15 ms
14 ms
13 ms
6N 7N 8N 9N 10 11 12
ode ode ode ode Nod Nod Nod
s s s s es es es
Cassandra Riak
Figure: S2 Result
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 23 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
25. Evaluation of
Virtual
Clusters
Performance
Conclusions
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito Dynamic architecture enabled:
Vieira and
Jorge G. • Performance scaling
Barbosa
• Cluster duplication
Introduction • Redundancy creation
Hypothesis • Increase infrastructure’s capabilities by using non-specialized HW
Goals
Performance losses
Methods
Results
• IO → Depends on cache’s efficiency, normally high (around 20%)
Conclusions
• Networking → High (Latency: 563% increase, Bandwidth: 70% decrease)
• CPU → Depends on the data locality:
• High → Low (≤ 5%)
• Low → High (≤ 35%)
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 24 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
26. Evaluation of
Virtual
Clusters
Performance
Future Work
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
Introduction Evaluate other setups
Hypothesis • Xen, VirtualBox, VMWare, etc.
Goals • Different IO modules/configurations
Methods • OpenStack, Nimbus, Eucalyptus, etc.
Results
Conclusions
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 25 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
27. Evaluation of
Virtual
Clusters
Performance
Q&A
on a Cloud
Computing
Infrastructure
Nuno
Cardoso, Tito
Vieira and
Jorge G.
Barbosa
Introduction
Hypothesis
Goals
Methods
?
Results
Conclusions
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 26 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011
28. Evaluation of
Virtual
Clusters
Performance
Backup
on a Cloud
Computing
Infrastructure 140 %
Nuno 100 %
Cardoso, Tito
Vieira and 60 %
Jorge G.
Gain
Barbosa 20 %
Introduction −20 %
Hypothesis −60 %
Goals −100 %
Methods −140 %
Results 1000 2000 4000 8000 16000 32000 46336
Conclusions
Problem Size
VM (resources x 2) HOST (resources x 2)
VM (resources x 4) HOST (resources x 4)
Figure: HPL performance gain by increasing processing power
Nuno Cardoso, Tito Vieira and Jorge G. Barbosa (Universidade do Clusters PerformanceEngenharia) Computing Infrastructure 27 / 26
Evaluation of Virtual Porto, Faculdade de on a Cloud November 4, 2011