Poster presented at PEARC20.
There is increased awareness and recognition that public Cloud providers do provide capabilities not found elsewhere, with elasticity being a major driver. The value of elastic scaling is however tightly coupled to the capabilities of the networks that connect all involved resources, both in the public Clouds and at the various research institutions. This poster presents results of measurements involving file transfers inside public Cloud providers, fetching data from on-prem resources into public Cloud instances and fetching data from public Cloud storage into on-prem nodes. The networking of the three major Cloud providers, namely Amazon Web Services, Microsoft Azure and the Google Cloud Platform, has been benchmarked. The on-prem nodes were managed by either the Pacific Research Platform or located at the University of Wisconsin – Madison. The observed sustained throughput was of the order of 100 Gbps in all the tests moving data in and out of the public Clouds and throughput reaching into the Tbps range for data movements inside the public Cloud providers themselves. All the tests used HTTP as the transfer protocol.
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
Demonstrating 100 Gbps in and out of the public Clouds
1. Demonstrating 100 Gbps in and out of the public Clouds
Igor Sfiligoi - University of California San Diego - isfiligoi@sdsc.edu
Most of the Cloud costs were covered by credits issued by Amazon, Microsoft and Google.
This work was partially funded by US National Science Foundation (NSF) under grants OAC-1941481, MPS-1148698, OAC-1841530, OPP-1600823, OAC-19044, and OAC-1826967.
Like it or not, public Clouds are gaining popularity
• Especially great for prototyping
• But great for compute spikes
and urgent compute activities, too
• When choosing which resources to use (and pay for)
knowing what to expect out of them is important
• The compute part of public Cloud resources is well documented
• The networking is instead hardly documented at all
• Yet networking can make a big difference
when doing distributed computing
Funding agencies are taking notice
• See NSF E-CAS award(s) as an example
• SDSC Expanse has Cloud explicitly mentioned
• NSF-funded CloudBank
• IceCube’s Cloud runs funded by an EAGER award
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 200 400 600 800 1000 1200 1400
Gbps
Number of compute instances
AWS
Azure
GCP
AWS Azure GCP
Asia – US West 100 Gbps 110 Gbps 940 Gbps
US East – West 440 Gbps 190 Gbps 1060 Gbps
EU – US East 460 Gbps 185 Gbps 980 Gbps
Peak Throughput observed between regions
of a single public Cloud provider
Peak Throughput observed in
a single public Cloud provider region
Networking inside
public Cloud providers plentiful
• Over 1 Tbps in a single region
• 100 Gbps to 1 Tbps between regions
Importing data into public Clouds
to feed compute instances
• Using multiplee parallel clients
• 100 Gbps doable on
production infrastructure
• Demonstrated with both storage
in the Pacific Research Platform and
at University of Wisconsin – Madison
ThroughputinBytespersecond
ThroughputinBytespersecond
Fetching data from public Clouds
• Fetching from their Object Storage
using multiple parallel clients
• 100 Gbps doable on
production infrastructure
(PRP using 20 nodes)
• Over 500 Gbps feeding
Cloud instances running
in multiple providers
AWS Azure GCP
US West 100 Gbps 120 Gbps 110 Gbps
US Central 100 Gbps 120 Gbps 110 Gbps
ThroughputinBytespersecond
Networking is a billed commodity
•$50/TB – $120/TB egress
• $20/TB - $90/TB data movement
between Cloud regions
• Ingress free
Fetching data from
Cloud storage
Charges on an account
used for benchmarking
Compute
and ingress
15% network waiver for
compute intensive workloads