Streaming Media West: Webrtc the future of low latency streaming

WebRTC 
The future (?) champion of
low latency
Dr Alex Gouaillard, CTO millicast.com

Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com
Streaming Media West - November 14, 2018 - Huntington Beach, California
https://medium.com/spankchain/spankchain-development-update-007-february-27-2018-414247c85445
Real-time? Why?
Well, have you ever tried to make love 
with 5s delay? (NSFW)

WebRTC ?

Historically: google gap analysis
• I own the net, or at least I have a toll on it
• I want everybody to do everything on the net
• What are people still using the desktop for they
cannot use the net for?

One-way Media Streaming pipeline
Media
Capture
Screen
Display
Encoding
Decoding
Segm
entation 
Fragm
entation
Reassem
bly
Encryption
Decryption
Transport
Internet
Internet
Transport

Media
Capture
Screen
Display
Encoding
Decoding
Segm
entation 
Fragm
entation
Reassem
bly
Encryption
Decryption
Transport
Internet
Internet
Transport
WebRTC STACK 
Media Engine
WebRTC STACK
Media Engine
In the Browser UA, No plugin, standard
Based on 20 years battle-tested VoIP techs (RTP) 
latency = ( RTT + 100 ms ) +

no need for Eng.
Web app is enough

Google Initiative, to start a standard
• 2009 - ON2 acquisitions (libVPx) for 100+ M
• 2010 - GIPS acquisition (libjingle => webrtc) 68M
• 2011 - Google Announce the WebRTC initiative

WebRTC Today
• Chromecast
• Youtube live
• Google’s Hangout, meet, allo, duo,
• Comcast Xinfinity
• Facebook messenger since 2012
• 17 billions video chats in 2017
• messenger is the 2nd most popular iOS app
• …
https://www.youtube.com/watch?v=n1gahO5S0Bg

Theoretical Streaming
( with WebRTC )

Media
Capture
Screen
Display
Encoding
Decoding
Segm
entation 
Fragm
entation
Reassem
bly
Encryption
Decryption
Transport
Internet
Internet
Transport
Media Engine  Media Engine

Optimisations for pre-recorded content Streaming
Media
Capture
Screen
Display
Encoding
Decoding
Segm
entation 
Fragm
entation
Reassem
bly
Encryption
Decryption
Transport
Internet
Internet
Transport
Pre-Encoded
C
ontent
D
ual-pass
encoding
Pre-C
hunk’ed 
uploaded
C
ontent
internet cache
protected
by
buffer
C
an
O
ptim
ise
p2p 
ex: peer5, stream
root.io

Problems / Solutions
• Large Bandwidth fluctuations
• Heterogeneous (receiving / viewer clients)
• Have several resolutions of the original available, and a way
to switch between them.
• multiple encoders (waste info in low rez)
• file chunks => switch in receiving application (slower, longer path bigger
chunks, buffers)
• WebRTC simulcast => switch in SFU
• layered encoder (SVC) => switch in SFU

Bandwidth Adaptive Media Streaming pipeline
Media
Capture
Screen
Display
Encoding
Decoding
Segm
entation 
Fragm
entation Reassem
bly
Encryption Decryption
Transport
Internet
Internet
Transport
MediaDistributionServer
Media Engine 
Media Engine Media Engine 
Media Engine

in practice - the usual
MediaCapture
Screen
Display
Media Engine (send) 
Media Engine (rec) 
ME
(send) 
ME 
(rec) 
Player
Transcoder  
possibly hosted and distributed 
e.g. BitMovin
Source
RTMP HLS

in practice - webrtc end-to-end
MediaCapture
Screen
Display
Media Engine (rec) 
PlayerSource
No storage
1 less enc/dec
= 50% load

• Ports, multiplexing, Nat and firewall firewall,
transport protocol, …
• discovery, handshake, streaming
• file-based => “tunnel” over HTTP and use URIs
(add delay)
• WebRTC => Dynamic transport allocation and NAT
traversal with ICE

• Network quality adaptation (jitter and packet loss)
• file-based => buffer (Add delay)
• WebRTC => use RTP (jitter buffer, NACK, PLI, RTX, RED, FEC) 
• Small Network bandwidth adaptation
• file-based => buffer (Add delay), and file-based (add delay)
• WebRTC => bitrate adaptation based on RTP bandwidth evaluation 
• Multistreams congestion control and multiplexing
• file-based => some multiplexing included, no CC
• WebRTC => rtcp-mux, BUNDLE, and transport-CC

Practical Streaming
( with WebRTC )

Simulcast, codecs, SVC support
10/2018 10/2018
04/2018 06/2018
xx/2018
Millicast / CoSMo software contributions 
to the browser vendor code base

Simulcast, codecs, SVC support

webrtc.org the default client side stack

CallStats.io: Network Instrumentation and Stats: Verify

KITE Interop SE Grid - Browser configs

WebRTC Testing: Interesting scientific publications
NARVAL, A No-Reference Video Quality Tool for Real-Time Communications, 
Augustin Lemesle, Alexis Marion, Ludovic Roux and Alexandre Gouaillard
in Proceedings of Human Vision and Electronic Imaging, Burlingame, California, USA, January
2019

1 OS
“Kite” Interop. mode: AppRTC Design
Conductor
Dispatcher
2 browsers, AppRTC-Test
list of N configs
Validate Config, against SE Grid
Interop. mode: Compute list of tests, i.e. pairs of BrowserObj
AppRTC-Test,
N^2 pairs of BO Smart Load Balancing of the N^2 pairs,
Aggregate results and send to Dashboard
1 2
AppRTC-Test,
1 pair< BO >
Run tests and instruments 2 browsers 
Collect/Compute result
Pluggable AppRTC-Test
Connect to appRTC URL
wait for “connected ||
completed”
or timeout
Pluggable Hybrid
SE Grid
local-Grid
Rem-BB
Rem-BaaS
(SauceLabs)
Rem-BaaS
(BrowserStack)
Rem-VM (AWS)
Test Runner
Validate
Configs
Pluggable Viz Dashboard
Visualize and Interact with Results
Test Runner
Test Runner
Test Runner

Comparative Study of WebRTC Open Source SFUs for Video Conferencing, 
Emmanuel André, Nicolas Le Breton, Augustin Lemesle, Ludovic Roux and Alex. Gouaillard
in Proceedings of IIT Real-Time Communications, Illinois Institute of Technology, Chicago, USA, October 2018

WebRTC Testing: Interesting scientific publications (1/2)

WebRTC Testing: Streaming: CPU footprint

WebRTC Testing: Streaming: Bitrate Adaptation

WebRTC Testing: Bitrate Adaptation

WebRTC at scale: cascading, not clustering
• Vidyo 
https://vidyo.io/blog/features/vidyo-io-delivers-massive-
scalability-maintaining-reliability-quality-cascading-sfus/
• Jitsi: https://webrtchacks.com/sfu-cascading/

WebRTC at scale: cascading, not clustering

Conclusion - the future
• Webrtc is already used by giants, accumulating
billions of minutes of audio and video annually.
The streaming industry seems to be slightly late
to the game.
• The standards are already thinking about Next
Version: QUIC, end-to-end encryption, Machine
learning, AV1, …..

Oct, 31st 2018 
ﬁrst public webrtc call 
with AV1
Oct, 31st 2018 
happy halloween eﬀect.

Thank You

The WebRTC stack
Media
Capture
Screen
Display
Encoding Decoding
Raw Media
Bitstream
Segmentation 
Fragmentation
Reassembly
Jitter bufferRTP packet
RTCP packetpayload
S & R S & R
SRTP packet
TCP / UDP / TLS / DTLS / QUIC packets
IPv4,6 packets
Transport
Internet
link layer and below (3)
Dynamically 
allocated 
through ICE

The HLS / MPEG-DASH way
Media
Capture
Screen
Display
Encoding Decoding
Raw Media
Bitstream
Segmentation 
Fragmentation
Reassembly
fmp4, MPEG2 TS, CMAF
S & R S & R
AES-128
HTTP(s) => TCP(TLS) packets
IPv4,6 packets
Transport
Internet

The WebRTC stack
Media
Capture
Screen
Display
Encoding Decoding
Raw Media
Bitstream
Segmentation 
Fragmentation
Reassembly
Jitter buffer
RTP packet: media transport
RTCP packet: Bitrate Adaptation, Bandwidth estimation, Congestion Control
payload
S & R S & R
SRTP packet: authenticated packet with encrypted content
TCP / UDP / TLS / DTLS / QUIC packets
IPv4,6 packets
Transport
Internet
All Media Specific
features are in RTP, 
 
Not in TRANSPORT !

Streaming Media West: Webrtc the future of low latency streaming

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Streaming Media West: Webrtc the future of low latency streaming

Similar to Streaming Media West: Webrtc the future of low latency streaming (20)

More from Alexandre Gouaillard

More from Alexandre Gouaillard (9)

Recently uploaded

Recently uploaded (20)

Streaming Media West: Webrtc the future of low latency streaming