Kixs & ふくあず 合同企画

2. {
“名前” : “真壁 徹(まかべ とおる)”,
“所属” : “日本マイクロソフト株式会社”,
“役割” : “クラウド ソリューションアーキテクト”,
“経歴” : “大和総研  HP Enterprise”,
“特技” : “クラウド & オープンソース”
}

6. 稼働中
アナウンス済み / 構築中
36 の地域でサービス中、40 の地域まで拡大予定
世界に広がるインフラストラクチャー
 100カ所以上のデータセンター
 米国 国防総省 (US DoD) も採用
(2017年 8月1日現在)
https://azure.microsoft.com/ja-jp/regions/

8. https://blogs.technet.microsoft.com/hybridcloud/2016/05/26/microsoft-and-facebook-
to-build-subsea-cable-across-atlantic/
https://azure.microsoft.com/ja-jp/blog/microsoft-invests-in-subsea-cables-to-
connect-datacenters-globally/
160Tbps 80Tbps

9. 12.88ミリ秒18.92ミリ秒
9.00ミリ秒

11. Colocation Density
2.0+Power Usage
Effectiveness (PUE) 1.4–1.6PUE
Discrete servers
Capacity
20year technology
Rack
Density &deployment
Minimized resource impact
Generation 1 Generation 2
Containment Modular Hyper-scale
1.2–1.5PUE
1.12–1.20PUE 1.07–1.19PUE
Containers, PODs
Scalability &sustainability
Air&waterEconomization
Differentiated SLAs
Deployment Areas &ITPACs
Nomore traditional IT
Right-sized
Faster time-to-market
Outsideaircooled
Fully integrated
Resilientsoftware
Common infrastructure
Operational simplicity
Flexible &scalable
Generation 3 Generation 4 Generation 5

13. http://natick.research.microsoft.com/

16. ” Albert Greenberg, Distinguished Engineer Director of Networking, Microsoft, SIGCOMM 2015

17. T2-1-1 T2-1-2 T2-1-8
T3-1 T3-2 T3-3 T3-4
Row Spine
T2-4-1 T2-4-2 T2-4-4Data Center Spine
T1-1 T1-8T1-7…T1-2
… …
Regional Spine
…
T1-1 T1-8T1-7…T1-2 T1-1 T1-8T1-7…T1-2
Rack …T0-1 T0-2 T0-20
Servers
…T0-1 T0-2 T0-20
Servers
…T0-1 T0-2 T0-20
Servers
Microsoft's Production Configurable Cloud” Mark Russinovich, Chief Technology Officer, Microsoft Azure, SCS Distinguished Lecture, 11/15/2016

19. • Microsoftがオープンソースとし
て公開したL3スイッチソフト
ウェア
• ちょうど1年前のOCPで発表
• OCP/ONIE対応ハードウェアで動
く
http://files.opencompute.org/oc/public.php?service=files&t=726804ae4b1309d606159b65a09b4124

20. http://files.opencompute.org/oc/public.php?service=files&t=726804ae4b1309d606159b65a09b4124

23. Virtualization
Linux
Guest
Windows
Guest

24. Virtualization
vCPU
Nested Virt
Guest VM

25. Virtualization
vCPU
Linux
Guest
vCPU
Hyper-V
Isolation
vCPU
Windows
Guest

26. Soon Soon Soon

27. Application
Platform
Orchestrator
Instance
Azure Infrastructure

28. https://azure.microsoft.com/ja-jp/services/virtual-machines/sap-hana/

30. “The problem I have right now? It is supply chain. I am not so worried about technology. We have our Open Cloud
Server, which I think is very compelling in that it offers some real economic capabilities. But I have got to nurture my
supply chain because traditionally we bought from OEMs and now we are designing with ODMs so we can take
advantage of prices and lower our overall costs. So I am moving very, very quickly to build out new capacity, and I
want to do it in a very efficient and effective way and it is really about the commoditization of the infrastructure.”
( https://www.nextplatform.com/2016/09/26/rare-tour-microsofts-hyperscale-datacenters/ )
Rick Bakken, Sr. Director, Data Center Evangelism, Microsoft

35. https://github.com/opencomputeproject/Project_Olympus

37. https://schd.ws/hosted_files/ocpussummit2017/f2/OCP17%20Workshop_Microsoft%20Project%20Olympus%20Servers_3_8_2017.pdf

39. https://schd.ws/hosted_files/ocpussummit2017/f2/OCP17%20Workshop_Microsoft%20Project%20Olympus%20Servers_3_8_2017.pdf

40. A New Era in the Datacenter (the launch of AMD EPYC. Footage taken from the launch event in Austin, Texas on June 20th, 2017)

41. • Cavium ThunderX2
• Qualcomm Centriq 2400
• モバイルデバイス向けに数が伸
び続けるARMへの期待
• 電力あたり性能
• エコシステム
• Windows ServerをARM対応中
• まずはAzureやマイクロソフトの
オンラインサービスなど、ユー
ザーが意識しないところで採用
• ソフトウェアとしての販売は未定
https://schd.ws/hosted_files/ocpussummit2017/f2/OCP17%20Workshop_Microsoft%20Project%20Olympus%20Servers_3_8_2017.pdf
http://files.opencompute.org/oc/public.php?service=files&t=5e232f151532e2af3b9754af4ca79f0b

42. http://files.opencompute.org/oc/public.php?service=files&t=14ab3cf25170b7a0a439e11a3d818c96
https://azure.microsoft.com/en-us/blog/ecosystem-momentum-positions-microsoft-s-project-olympus-as-de-facto-open-compute-standard/
• “Hyperscale GPU Accelerator”
• NVIDIAとの共同開発
• Tesla P100 SMX2 x8 & NVLink
• 4シャーシまで拡張可
• Internal PCIe Fabric Interconnect
• 16サーバーまで共用可

43. http://files.opencompute.org/oc/public.php?service=files&t=14ab3cf25170b7a0a439e11a3d818c96

44. http://files.opencompute.org/oc/public.php?service=files&t=14ab3cf25170b7a0a439e11a3d818c96

45. http://files.opencompute.org/oc/public.php?service=files&t=14ab3cf25170b7a0a439e11a3d818c96

46. http://files.opencompute.org/oc/public.php?service=files&t=309c5b1d58539b56a4cd5d499c5b3a85
High Density Flash Expansion “FX-16”

47. http://files.opencompute.org/oc/public.php?service=files&t=309c5b1d58539b56a4cd5d499c5b3a85

49. FPGAs
EFFICIENCY
Control
Unit
(CU)
Registers
Arithmetic
Logic Unit
(ALU)
+
+
+
+
+
+
+
FLEXIBILITY
CPUs GPUs
ASICs

50. DRAM
Controller
USB Controller Ethernet Controller
DSP
Slice
RAM
RAM
DSP
Slice
CPU
CPU

51. FPGA
FPGA
Data
Instruction
Instruction
Instruction
Data
Instruction
Instruction
Instruction
CPU
CPU
Instruction

53. Catapult v0
Catapult v1
Scale v1
Catapult v2
2011 2012 2013 2014 2015 2016 …
Ignite unveiling
Production

55. WCS Gen4.1 Blade with NIC and Catapult FPGA
Catapult v2 Mezzanine card

56. “ハードウェア実装によ
る性能とソフトウェア
実装による柔軟性を両
立できないか?
サーバーNICは年々広帯域化:
1G  10G  40G  50G  100G
VMを集約することでコストを下げられるが、同時に帯
域が必要になる
CPUがネットワーク処理で飽和してしまう
新しいネットワーク機能の実装:
BYO IP, BYO Topology, BYO Appliance
今後も多様な処理、役割をSDN、仮想ネットワーク上で
実現したい
新しいアイデアを迅速、柔軟に実装したい (ASICでは時
間がかかりすぎる、リスクが高すぎる)

57. FPGAにネットワーク関連処理を実装
すでにBing (Catapult)でFPGAの知見があった
Generic Flow Tables (GFT)に従いパケット処理
パケットの処理ルールをGFTに書く
2015年後半以降に導入したサーバーに載っ
ている
暗号化、QoS、圧縮などにも使われている
Host
SmartNIC
FPGA
ToR
NIC ASIC
SmartNIC
CPU

58. VM
VFP
Southbound API
GFT Offload API (NDIS)
VMSwitch
Northbound API
GFT
Table
First Packet
GFT Offload Engine
50G
QoSCrypto RDMA
GFT
Transposition
Engine
REWRITE
SLB Decap SLB NAT VNET ACL Metering
ControllerControllerController
Encap
SmartNIC
DNATDecap Allow Meter
Rule Action
* Meter
Rule Action
* Allow
Rule Action
* Rewrite
Rule Action
* DNAT
Rule Action
* Decap
Flow Action
1.2.3.1->1.3.4.1,
62362->80
Decap, DNAT,
Rewrite, Meter
Flow Action
1.2.3.1->1.3.4.1,
62362->80
Decap, DNAT,
Rewrite, Meter

59. 従来方式 Accelerated networking
(FPGA & SR-IOV & VFP)
VM 1 VM 2
Virtual switch
Physical
server 1
Physical switch
Virtual switch
Physical
server 2
Virtual
Network VM 1 VM 2
Physical switch
Virtual
Network
4Gbpsで
頭打ち
25Gbpsを
実現

60. ToR
FPGA
NIC
Server
FPGA
NIC
Server
FPGA
NIC
Server
FPGA
NIC
Server
CS0 CS1 CS2 CS3
ToR
FPGA
NIC
Server
FPGA
NIC
Server
FPGA
NIC
Server
FPGA
NIC
Server
SP0 SP1 SP2 SP3
L0
L1/L2
Microsoft's Production Configurable Cloud” Mark Russinovich, Chief Technology Officer, Microsoft Azure, SCS Distinguished Lecture, 11/15/2016
October 15, 2016

61. Credits
Virtual Channel
Data
Header
Elastic
Router
(multi-VC
on-chip
router)
Send Connection Table
Transmit State Machine
Send
Frame
QueueConnection
Lookup
Packetizer
and
Transmit
Buffer
Unack’d
Frame
Store
Ethernet
Encap
Ethernet
Decap
40G
MAC+PHY
Receive Connection Table
Credits
Virtual Channel
Data
Header
Depacketizer
Credit
Management
Ack Receiver
Ack Generation
Receive State Machine
Solid links show Data flow, Dotted links show ACK flow
Datacenter
Network
Microsoft's Production Configurable Cloud” Mark Russinovich, Chief Technology Officer, Microsoft Azure, SCS Distinguished Lecture, 11/15/2016

62. Microsoft's Production Configurable Cloud” Mark Russinovich, Chief Technology Officer, Microsoft Azure, SCS Distinguished Lecture, 11/15/2016

63. New 50GbE SmartNIC for Project Olympus
(Announced at OCP 2017)

64. Management
Fabric
Hardware
(FPGA)
Super Low-
latency
Network

65. Traditional software (CPU) server plane
QPI CPUCPU
QSFP
TOR40Gb/s
Web search
ranking

66. Web search
ranking
Traditional software (CPU) server plane
QPICPU
QSFP
40Gb/s ToR
FPGA
CPU
40Gb/s
QSFP QSFP
Hardware acceleration plane
Web search
ranking
Deep neural
networks
SDN offload
SQL

67. CPU FPGA CPU FPGA

68. CPU FPGA CPU FPGA