Aimed at anyone starting their journey with providing ADSL/FTTC/SoGEA/GFast/FTTP via wholesale L2TP. Will touch on ISO/OSI layers 1-7 + 8 + 9

1. L2TP 101
ON-RAMP TO CONSUMING
WHOLESALE BROADBAND SERVICES
https://faelix.link/netmcr57

2. About Marek
Stuff I do:
CTO @FAELIX – https://faelix.net/
PC @uknof – https://uknof.uk/
Crew @net_mcr – https://www.netmcr.uk/
Trail of SSIDs in my wake: "AS41495 Faelix Limited"
Me — @maznu – @NetworkMoose

3. This Talk
Aimed at anyone starting their journey with
providing ADSL/FTTC/SoGEA/GFast/FTTP via
wholesale L2TP.
Will touch on ISO/OSI layers 1-7 + 8 + 9.

4. L1: THE LAST MILE

5. WLR
router
modem
ONT
etc
street furniture
exchange
“telephone
network”
ISP
box
ISP
edge
device
W
LR

6. ADSL
router
modem
ONT
etc
street furniture
exchange
“telephone
network”
ISP
box
ISP
edge
device
Fibre
Copper
W
LR

7. FTTC
router
modem
ONT
etc
street furniture
exchange
“telephone
network”
ISP
box
ISP
edge
device
Fibre
Copper
W
LR

8. SoGEA
router
modem
ONT
etc
street furniture
exchange
“telephone
network”
ISP
box
ISP
edge
device
Fibre
Copper

9. GFast
router
modem
ONT
etc
street furniture
exchange
“telephone
network”
ISP
box
ISP
edge
device
Fibre
Copper
W
LR

10. FTTP
router
modem
ONT
etc
street furniture
exchange
“telephone
network”
ISP
box
ISP
edge
device
Fibre
VoIP

11. FTTP: The ONT
Copper demarcation is “BT Master Socket”
Fibre demarcation is “The ONT”
PON fibre in, ethernet out (and maybe FXS for VoIP)
Some ONTs have multiple ethernet ports
Separate customer services, multiple providers
Total bandwidth still limited by the PON

12. FTTP: Single-Port ONT
https://order.faelix.net/product/broadband/availability/zen/

13. FTTP: Multi-Port ONT
https://order.faelix.net/product/broadband/availability/zen/

14. L2: BROADBAND

15. Overview
router
modem
ONT
etc
street furniture
exchange
ISP
edge
device
PPP
“telephone
network”
ISP
box

16. Overview
router
modem
ONT
etc
street furniture
exchange
wholesale
“provider”
network
ISP
LNS
ISP
edge
device
PPPoE
L2TP

17. L2.5: L2TP

18. L2TP (v2)
L2TP = Layer-2 Tunnelling Protocol
Typically IPv4 UDP port 1701
20 (IP) + 8 (UDP) + 12 (L2TP) bytes of headers
Multiple sessions within one tunnel
Tunnels can be authenticated with secret
PPP sessions can be authenticated
Can add/drop L2TP sessions in and out of tunnels

19. Terminology
LNS = L2TP Network Server
LAC = L2TP Access Concentrator
LTS = L2TP Tunnel Switch (Cisco “L2TP Multihop”)
RADIUS = Remote Authentication Dial-In User
Service

20. Overview
router
modem
ONT
etc
street furniture
exchange
wholesale
“provider”
network
ISP
LNS
ISP
edge
device

21. Overview
ISP
edge
LTS
LTS
LAC
LNS
PPPoE
L2TP
RADIUS
AAA

22. LAC and LNS
LAC “concentrates” (aggregates) customers
Based on the authenticating user’s realm it will try
to create an L2TP tunnel and session to your LNS
Might use RADIUS steering to determine LNS’ IPs
and L2TP secrets
Or might have a static per-realm configuration
And now you’re running PPP end-to-end!

23. Steering
ISP
edge
LTS
LTS
LAC
LNS
PPPoE
RADIUS

24. Steering
ISP
edge
LTS
LTS
LAC
LNS
RADIUS
Access-Request

25. Steering
ISP
edge
LTS
LTS
LAC
LNS
Access-Accept
Tunnel-Server-Endpoint := 192.0.2.1
Tunnel-Password := hunter2
Tunnel-Type := L2TP
Tunnel-Medium-Type := IP
RADIUS

26. Steering
ISP
edge
LTS
LTS
LAC
LNS
L2TP
RADIUS

27. Steering
ISP
edge
LTS
LTS
LAC
LNS
RADIUS
Access-Request

28. Steering
ISP
edge
LTS
LTS
LAC
LNS
RADIUS
Access-Accept
Framed-Protocol := PPP
Framed-Address := 198.51.100.1
Framed-Netmask := 255.255.255.255
Framed-IPv6-Prefix: 2001:db8::/32
NAS-Port-Id: ppp123
PPPoE
L2TP

29. Steering
ISP
edge
LTS
LTS
LAC
LNS
RADIUS
PPP

30. L3-7: SHOW CONFIG

31. LNS Implementations
Cisco IOS-pretty-much-everything
Juniper JunOS, Nokia SR OS, etc
MikroTik RouterOS
VyOS 1.3 (accel-ppp)
Ubiquiti EdgeOS (can only LNS with IPsec)
Linux: accel-ppp, xl2tpd, l2tpnsd

32. MikroTik RouterOS: PPPoE
/interface ethernet
set [find name=ether1] mtu=1508 l2mtu=1540
/interface pppoe-client
add disabled=no name="pppoe-access"
user=“user@realm” password="password"
max-mru=1500 max-mtu=1500
use-peer-dns=yes add-default-route=yes
profile=default allow=pap,chap
keepalive-timeout=60
interface=ether1

33. VyOS: LNS
interfaces {
ethernet eth1 {
mtu 9000
offload {
sg
}
}
}
protocols {
bgp … {
}
}

34. VyOS: LNS
vpn {
l2tp {
remote-access {
ccp-disable
client-ip-pool {
start 100.64.0.1
stop 100.64.0.254
}
client-ipv6-pool {
delegate 2001:db0:44f3::/48 {
delegation-prefix 56
}
prefix 2001:db0:44f3:ff00::/56 {
}
}
}
}
}
vpn {
l2tp {
remote-access {
gateway-address 203.0.113.1
lns {
shared-secret hunter2
}
mtu 1500
name-server 9.9.9.9
name-server 1.1.1.1
outside-address 192.0.2.2
}
}
}

35. vpn {
l2tp {
remote-access {
authentication {
local-users {
username user@realm {
password password
static-ip 203.0.113.2
}
}
mode local
mppe deny
require chap
require mschap
require mschap-v2
require pap
}
}
}
}
VyOS: LNS
vpn {
l2tp {
remote-access {
authentication {
mode radius
radius {
server 198.51.100.1 {
key letmein
}
server 198.51.100.2 {
key letmein
}
source-address 192.0.2.2
}
}
}
}
}

36. MikroTik RouterOS: LNS
/routing bgp instance …
/routing bgp peer …
/ip address …
/interface ethernet
set [find name=ether1] mtu=1560 l2mtu=1600

37. MikroTik RouterOS: LNS
/ppp l2tp-secret
add address=192.0.2.0/29 secret=hunter2
/ppp profile
add name=LNS dns-server=9.9.9.9,1.1.1.1
local-address=203.0.113.1 only-one=yes
use-compression=no use-encryption=no use-mpls=no
/interface l2tp-server server
set enabled=yes authentication=pap,chap
caller-id-type=number default-profile=LNS
max-mru=1500 max-mtu=1500

38. MikroTik RouterOS: LNS
/ppp secret
add name=user@realm password=password
profile=LNS remote-address=203.0.113.2 service=l2tp
remote-ipv6-prefix=2001:db8::/112
/radius
add address=198.51.100.1 secret=letmein service=ppp
add address=198.51.100.2 secret=letmein service=ppp
/ppp aaa
set use-radius=yes

39. FreeRADIUS: Steering
update {
reply:Tunnel-Server-Endpoint:0 = “192.0.2.1”
reply:Tunnel-Password:0 = "hunter2"
reply:Tunnel-Type:0 = L2TP
reply:Tunnel-Medium-Type:0 = IP
control:Auth-Type = "Accept"
}
Full write-up at faelix.net/news (includes ExaBGP
and service tests for HA)

40. FreeRADIUS: AAA
Actually nothing clever required!
We added a feature to our setup:
user+steer@realm
treated as user@steer.realm for session steering
but treated as user@realm for auth

41. user+steer@realm
Steer sessions from user-side to specific LNSs
One tunnel to London, another to Manchester
BGP route servers to receive nearby IX CDN prefixes
Anycast DNS helps CDNs to serve traffic locally
Will be giving a talk about this at LINX on 31st March

42. L8+9: PROVIDERS

43. Experience: Enta
Available in: MA1, THN, THW, THE, LD8…
Seemed easy to onboard with
Were still Mbit/sec charges on some packages
Else strong expectation to achieve MoQ
BTW and Enta LLU
NB: we didn’t finish onboarding with Enta, but might
complete this in future if demand/requirement

44. Experience: ICUK
Available in: THN, THW, LD8
Pretty easy to on-board with: sent an email
Apply for your OFCOM RID before starting
Deposit required, plus some setup charges
One realm per customer, statically steered
One 1G NNI (unless you reach volume targets based
on lines, or pay monthly for extra NNI)
BTW and TTB

45. Experience: Zen
Available in: MA1, THN, THE… LON1/2/3?
Took ages to get them to talk to us
Dedicated onboarding, weekly progress meetings
Unlimited realms, supports RADIUS steering
Expectation of multiple 10G NNIs
“Want to see a sales path to hundreds of tails in Y1”
BTW and Zen LLU (and TTB, but not on wholesale)

46. Experience: APIs
ICUK’s API is easy for availability searches and
ordering (albeit slightly unusual authentication)
Zen’s API doesn’t like User-Agent: /.*python.*/
ICUK’s API for everything: WLR and broadband
Zen’s API only for broadband, can’t choose network
WLR is third-party, doesn’t seem to be API?

47. Experience: Price
£
Mbit/sec
ADSL FTTC GFast FTTP
10
20
30
60
BTW via ICUK
BTW via ICUK
Market A
TTB via ICUK
Zen BTW
Market A
Zen

48. Wholesale Line Rental
The “copper pair” required for ADSL, FTTC, GFast
Approximately £9-11/month (plus calls)
“Naked” FTTC = SoGEA (no telephone line)
SoGFast exists (but not always productised)
Services are slightly cheaper overall
FTTP has no copper pair, all fibre, VoIP telephony

49. ASK ME
ABOUT
BROADBAND
E: marek @ faelix . net
T: @maznu
T: @faelix
W: https://faelix.net/
https://faelix.link/netmcr57