AWS Direct Connect provides a more consistent network experience for accessing your AWS resources, typically with greater bandwidth and reduced network costs. This session dives deep into the features of AWS Direct Connect, including public and private virtual Interfaces, Direct Connect Gateway, global access, local preference communities, and more.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Direct Connect: Deep Dive
Justin Davies
Solutions Architect
AWS/Solutions Architecture
N E T 4 0 3

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s going on here?
policy-options
policy-statement TO-AWS
term tag-aws
from
route-filter 0.0.0.0/0 exact;
then
community add TAG-TO-AWS;
accept;
community TAG-TO-AWS-HIGH-PREF members 7224:7300;

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Level set—review
New features and functionality
Route manipulation and traffic engineering
How is AWS Direct Connect billed?
How to manage hybrid DNS scenarios over
AWS Direct Connect
Architectural best practices and resiliency

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Amazon Virtual Private Cloud (Amazon VPC)
On-premises

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Availability Zone
On-premises

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Availability Zone
On-premises

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Subnet
On-premises

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Subnet
Virtual private
gateway
On-premises

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Subnet
Virtual Private
Gateway
On-premises

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Subnet
Virtual Private
Gateway
Direct
Connect On-premises

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect On-premises
Customer
backbone
Amazon

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect On-premises
Customer
backbone
Amazon

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect
Amazon
…
Public
Private

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect
Amazon
…
Public
Private

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect
1G, 10G,

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Private VIF
Private
Virtual Private
Gateway
Physical connection
VLAN ID
VIF name & owner
On-prem ASN
*AWS ASN

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Private VIF
Private
Virtual Private
Gateway
Physical connection
VLAN ID
VIF name & owner
On-prem ASN
*AWS ASN
50 VIFs

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF
…
Public
Public VIF
Physical connection
VLAN ID
VIF name & owner
On-prem ASN
Public peer IPs (v4)

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF
…
Public
Public VIF
Physical connection
VLAN ID
VIF name & owner
On-prem ASN
Public peer IPs (v4)

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
”Home” region
https://aws.amazon.com/directconnect/features/
us-east-1 us-west-2

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Do I need to have a BGP session for every VPC?

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can I connect to VPCs outside of my “home” region?

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can I connect to VPCs outside of my “home” region?

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can I reduce my BGP peers and simplify connectivity?

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So what is a Direct Connect Gateway?

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct ConnectYou specify:
“name”

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway
1
2
10
Attached

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway
Account 1
Account 1
Account 2
Direct Connect

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So how does this scale?

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Account 1
Account 1
Account 1
Direct Connect
Attach 10

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Account 1
Account 1
Account 2
Direct Connect
Attach 10

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Account 1
Account 1
Account 2
Direct Connect
Attach 10

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Account 1
Account 1
Account 2
Direct Connect

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do routes work?

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do routes work?

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before Logical Redundancy Direct Connect
CustomerDirect Connect
Device

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before Logical Redundancy Direct Connect
CustomerDirect Connect

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before Logical Redundancy Direct Connect

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logical Redundancy (NEW) Direct Connect
CustomerDirect Connect
Direct Connect

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logical Redundancy (NEW) Direct Connect
Customer
Direct Connect

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How does this change my
physical redundancy?

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logical & Physical Redundancy Direct Connect
CustomerDirect Connect
Direct Connect

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Is logical redundancy available?

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Redundant BGP Sessions

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

66. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

67. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

68. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

69. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
West - DC
East
West
172.16.0.0/16
65001, 65001, 65001
172.16.0.0/16
65001, 65001
us-east-1

70. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
West - DC
East
West
172.16.0.0/16
65001, 65001, 65001
172.16.0.0/16
65001, 65001
*Preferred route
leaving AWS
us-east-1

71. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
West - DC
East
West
172.16.0.0/16
65001, 65001, 65001
172.16.0.0/16
65001, 65001
172.16.0.0/16
65001
*Preferred route
leaving AWS
us-east-1

72. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
West - DC
East
West
172.16.0.0/16
65001, 65001, 65001
172.16.0.0/16
65001, 65001
172.16.0.0/24
65001
*Preferred route
leaving AWS
us-east-1
*Longest prefix match

73. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BGP communities & local—preference

74. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF communities—Controls your prefix scope

75. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF communities—Controls AWS prefix scope

76. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Private VIF communities: AWS egress local-pref

77. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
West - DC
East
West
172.16.0.0/16
65001, 65001, 65001
172.16.0.0/16
65001, 65001
172.16.0.0/16
65001
*Preferred route
leaving AWS
us-east-1

78. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
West - DC
East
West
172.16.0.0/16
7224:7100 (low)
65001, 65001, 65001
172.16.0.0/16
7224:7100 (Low)
65001, 65001
172.16.0.0/16
7224:7300 (high)
65001
*Preferred route
leaving AWS
us-east-1

79. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Applying communities to prefixes
policy-options
policy-statement TO-AWS
term tag-aws
from
route-filter 0.0.0.0/0 exact;
then
community add TAG-TO-AWS;
accept;
community TAG-TO-AWS-HIGH-PREF members 7224:7300;
Juniper example

80. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Applying communities to prefixes
ip bgp-community new-format
ip prefix-list TAG-TO-AWS permit 0.0.0.0/0 le 32
route-map TO-AWS permit 10
match ip address prefix-list TAG-TO-AWS
set community 7224:7300
router bgp 65400
address-family ipv4
neighbor 169.254.221.5 send-community
neighbor 169.254.221.5 route-map TO-AWS out
Cisco example

81. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

82. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
I manage the network.
I’m not sure what all these VPCs are really doing.
How does billing work?

83. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1G = $0.30/port hour
10G = $2.25/port hour
*All locations except Japan
Data-Transfer-OUT
Source: United States
VPC, S3, DDB …
Destination:
Switch, SUPERNAP
Las Vegas
$0.0200/GB Out
https://aws.amazon.com/directconnect/pricing/
Direct Connect Billing

84. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1G = $0.30/port hour
10G = $2.25/port hour
*All locations except Japan
Data-Transfer-OUT
Source: Ireland
(eu-west-1)
VPC, S3, DDB …
Destination:
Switch, SUPERNAP
Las Vegas
$0.0282/GB Out
https://aws.amazon.com/directconnect/pricing/
Direct Connect Billing

85. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect: Port cost
https://aws.amazon.com/directconnect/pricing/

86. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect: Data-transfer-out cost
https://aws.amazon.com/directconnect/pricing/

87. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What if I have multiple accounts?

88. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Account 1
Account 2
Account 3
Account 4
Organization (master payer account)
Direct Connect Billing
$

89. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Account 1
Account 2
Account 3
Account 4
Direct Connect Billing
$
Source account

90. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

91. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
I manage DNS servers on-premises today.
How can I resolve resources between
my VPC resources and on-premises?

92. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.0/24 (myvpc.com)
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
10.0.0.0/16 (mydc.com)

93. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Host one: Where is ”two.myvpc.com”
1
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com

94. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Host one: Where is ”two.myvpc.com”
2. Amazon Route 53: Oh, that’s 192.168.1.11
1
192.168.1.2
192.168.1.0/24 (myvpc.com)
2
mydc.com

95. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Host one: Where is ”two.myvpc.com”
2. Amazon Route 53: Oh, that’s 192.168.1.11
1
192.168.1.2
192.168.1.0/24 (myvpc.com)
2
mydc.com

96. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
1

97. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Conditional forward?
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
1
2

98. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Conditional forward?
3. I don’t know, can’t reach 192.168.1.2
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
1
2
X

99. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
Unbound
1

100. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Forward to Unbound
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
Unbound
1
2

101. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Forward to Unbound
3. Unbound forward to Route 53
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
Unbound
1
2
3

102. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Forward to Unbound
3. Unbound forward to Route 53
4. Reply to requester
192.168.1.2
192.168.1.0/24 (myvpc.com)
mydc.com
Unbound
1
2
3
4

103. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Route 53 Resolver
Primary
Availability
Zone 1
Secondary Tertiary
Availability
Zone 2
Availability
Zone 3

104. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com)
mydc.com
1
192.168.1.xyz

105. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Forward to AWS resolver192.168.1.0/24 (myvpc.com)
mydc.com
1
2
192.168.1.xyz

106. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Client: Where is ”two.myvpc.com”
2. On-prem DNS: Forward to AWS resolver
3. Reply to requester
192.168.1.0/24 (myvpc.com)
mydc.com
1
2
3
192.168.1.xyz

107. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1
192.168.1.0/24 (myvpc.com)
mydc.com
10.0.0.7
192.168.1.xyz
1. Host one: Where is ”client.mydc.com”

108. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1
192.168.1.0/24 (myvpc.com)
2
mydc.com
10.0.0.7
192.168.1.xyz
1. Host one: Where is ”client.mydc.com”
2. Route 53: Forward *.mydc.com to on-prem DNS

109. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1
192.168.1.0/24 (myvpc.com)
2
mydc.com
3
10.0.0.7
192.168.1.xyz
1. Host one: Where is ”client.mydc.com”
2. Route 53: Forward *.mydc.com to on-prem DNS
3. On-prem DNS: Oh, that’s 10.0.0.7

110. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
192.168.1.10
one.myvpc.com
192.168.1.11
two.myvpc.com
1. Host one: Where is ”client.mydc.com”
2. Route 53: Forward *.mydc.com to on-prem DNS
3. On-prem DNS: Oh, that’s 10.0.0.7
4. Reply to requester
1
192.168.1.0/24 (myvpc.com)
2
mydc.com
3
4
10.0.0.7
192.168.1.xyz

111. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

112. “Everything fails all the time.”
Werner Vogels
VP & CTO, AWS

113. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start with the application
Availability Zone 1
Availability Zone 2

114. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start with the application
us-east-1
us-west-2

115. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start with the application
us-east-1
us-west-2

116. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider the ingress and egress points
Availability Zone 1
Availability Zone 2

117. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider the ingress and egress points
Availability Zone 1
Availability Zone 2

118. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider the ingress and egress points
Direct Connect On-premises

119. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Know your traffic profile
Direct Connect On-premises

120. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Know your dependencies
Everything API

121. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact

122. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact
Guilty until proven innocent
Test it! Test it often!

123. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact
Guilty until proven innocent
Test it! Test it often!

124. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact
Guilty until proven innocent
Test it! Test it often!

125. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Justin Davies
@mrjustind

126. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.