Presented at the 12th ACM Workshop on Hot Topics in Networks (HotNets XII) Datacenters have traditionally been a collection of individual servers, each of which aggregates a fixed amount of compute, memory, storage, and network resources as an independent physical entity. Extrapolating from recent trends, we envisage that future datacenters will be architected in a drastically different manner: all computational resources within a server will be disaggregated into standalone blades, and the datacenter network will directly interconnect them. This is what we call "Disaggregated Datacenters". This presentation briefly sketches why and how this transition will happen. In particular, we focus on the role of network fabric in such disaggregated datacenters, as it will face a set of challenges brought by the new datacenter architecture. The paper can be found here: http://www.eecs.berkeley.edu/~sangjin/static/pub/hotnets2013_ddc.pdf

1. Network Support
for Resource Disaggregation
in Next-Generation Datacenters
Sangjin Han Norbert Egi Aurojit Panda
Sylvia Ratnasamy Guangyu Shi Scott Shenker
UC Berkeley Futurewei Technologies ICSI
1

2. Future datacenters will look
fundamentally different.
There will be no “servers”.
Like it or not.
2

3. Outline:
1. What will happen?
2. Why will it happen?
3. How will it happen?
3

4. Today: Server-Centric Architecture
Intra-server
Network
Datacenter
Network
4

5. Tomorrow: Resource-Centric Architecture
All resources are
individually addressable
Datacenter
Network
5

6. The Trends: Disaggregation
1. HP MoonShot
– Shared cooling/casing/power/mgmt for server blades
6

7. The Trends: Disaggregation
1. HP MoonShot
2. AMD SeaMicro
– Virtualized I/O
Datacenter
Network
7

8. The Trends: Disaggregation
1. HP MoonShot
2. AMD SeaMicro
3. Intel Rack Scale Architecture
8

9. The Trends: Disaggregation
1.
2.
3.
4.
HP MoonShot
AMD SeaMicro
Intel Rack Scale Architecture
Open Compute Project
9

10. Disaggregated Datacenter
Resource as a
standalone blade
Datacenter
network
10

11. Why will it happen?
: Extreme resource modularity
11

12. Benefits of Resource Modularity
1. Easier to build evolve
– Resources have different cycles/trends/constraints.
•
•
Tight integration in a server is a huge pain
E.g., “Memory capacity per core drops 30%
for every 2 years” [Lim et al., ISCA ’09]
– Disaggregation enables independent evolution
– The biggest driving force from vendor’s viewpoint
12

13. Benefits of Resource Modularity
1. Easier to build evolve
2. Fine-grained resource provisioning
– Current practice: replace/buy an entire
server, rack, or even datacenter.
– e.g., “I want just more processors, not servers!”
•
Go buy some CPU blades at Best Buy® and plug them in.
– e.g., “I want to try the new NVRAM technology!”
•
Again, go for it.
13

14. Benefits of Resource Modularity
1. Easier to build evolve
2. Fine-grained resource provisioning
3. Operational efficiency
– Datacenter as a single giant computer
– Higher utilization with statistical multiplexing
– (I will get back to this)
14

15. How will it happen?
: Incrementally and radically.
15

16. THIS IS NOT A CRAZY IDEA
Unified
Interconnect
• Do we need to change everything? NO.
– HW change is minimal.
– SW change is minimal, too.
16

17. HW Requires Minimal Modification
Resource as a standalone blade
• The internals don’t need to change.
• All we need is embedded network controller.
– They already have: QPI, HT, PCIe, SATA, …
– Can be very cheap
• E.g., a whole graphics card w/ 128Gbps for only $50
17

18. How About SW?
• Existing SW infrastructure heavily relies on
the concept of “server”
– We don’t want to rewrite it from scratch.
– How to utilize the “giant computer”?
VM
Disaggregated
VM
No modification for App/OS
Minor changes in VMM.
Much higher utilization!
18

19. Elastic VMs Achieve High Utilization!
Task
1
Task
2
Servers
Task
3
Task
4
19

20. Elastic VMs Achieve High Utilization!
40%
of
resources
are
wasted
Servers
20

21. Elastic VMs Achieve High Utilization!
Disaggregated
datacenter
1. No “server boundary”
2. Statistical multiplexing at a larger scale
3. Higher utilization!
40%
of
resources
are
wasted
vs.
Servers
21

22. (tasks vary greatly)
22

23. THIS IS NOT A CRAZY IDEA, part 2
• We don’t need to change everything.
– HW change is minimal.
– SW change is minimal, too.
• A unified network is plausible
– The intra-/inter-server networks can be unified.
– Bandwidth/latency requirements are within reach.
23

24. Two Different(?) Types of Network
Intra-server
Network
Inter-server
Network
24

25. Intra- vs. Inter-Server Networking
E.g., PCIe and 10GbE
Aren’t they two
different things?
Not really.
•
•
•
•
•
•
Serial
Point-to-point
Full duplex
Packet-switched
Variable packet size
Supports both message and
read/write semantics
No fundamental™ difference!
25

26. Making Memory Traffic Manageable
Registers
10,000 Gbps
1 ns
CPU
Cache
500 Gbps
50 ns
Memory
1-10 Gbps
50,000 ns
SSD
/
HDD
26

27. Making Memory Traffic Manageable
Registers
10,000 Gbps
1 ns
CPU
Cache
500 Gbps
50 ns
Local
memory
?? Gbps
?? ns
Remote
memory
1-10 Gbps
50,000 ns
A small amount of
local memory as a “cache”
SSD
/
HDD
27

28. Desirable Network Speed?
• A quick-and-dirty experiment
Emulated
Local memory
“remote” memory
Server
Core
Core
Core
Core
Artificial delay for
bandwidth/latency
28

29. Desirable Network Speed?
• 4 CPU cores, 8GB working set size
• GraphLab, memcached, Pig
• Findings (read the paper)
1. 10-40 Gbps is enough.
1. Feasible even today!
2. Average link utilization: 1-5Gbps
2. Latency matters.
29

30. WANTED: Low Latency
memcached with varying latency
10µs
latency,
20%
overhead
30

31. Research Questions
31

32. Questions
• Answered:
– How fast should it be? 10-40Gbps, 1-10µs.
• Unanswered:
–
–
–
–
–
Scalability?
Reliable transfer?
QoS?
Packet? Circuit?
…
32

33. 1. “Right” Scale of Disaggregation
• Disaggregation scale: where is the sweet spot?
TradiIonal
inter-­‐server
network
datacenter-scale
(flat)
chassis/rack/pod-scale
(two-tier)
33

34. 2. Realizing Low Latency
• It’s time for low latency [Rumble et al., HotOS ’11]
– “5-10µs latency is possible in the short term”
– “1µs round-trip times cannot be achieved off-processor”
• Congestion avoidance/control should be
“close to the metal”.
– A lot of research efforts are ongoing.
– Will they be still valid in disaggregated datacenters?
34

35. 3. Unified Scheduler
• We will need a unified scheduler:
– Job scheduler + network controller
VM1
VM2
VM1
VM1
VM2
35

36. 3. Unified Scheduler
• We will need a unified scheduler:
– Job scheduler + network controller
VM1
VM1
VM1
VM2
VM2
36

37. 3. Unified Scheduler
• We will need a unified scheduler:
– Job scheduler + network controller
VM1
VM1
VM1
VM2
VM2
37

38. Closing Remarks
• Disaggregated datacenter will be “the next big thing”
– Already happening. We need to catch up!
• We are working on a small-scale prototype.
– Disaggregated resource blades on existing SW/HW
• 40 CPUs
• 6 remote memory blades
• 8 GPU/NIC/storage blades
– PCIe as the “unified interconnect”
38