More Related Content Similar to 2016 February - WebRTC Conference japan - English (20) More from Alexandre Gouaillard (19) 2016 February - WebRTC Conference japan - English2. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• Faster ICE
• Ports optimizations
• Simulcast
• Others in webrtc 1.0
3. Early 2015, peer-to-peer webRTC Model
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
http://www.html5rocks.com/en/tutorials/webrtc/basics/
4. Signaling
server
Early 2015, peer-to-peer webRTC Model
PeerConnection
JavaScript APP(Sender) JavaScript APP (Receiver)
PeerConnection
Media
Stream
Data
Channel
CreateOffer
SetLocal
SetRemote
SetLocal
SetRemote
Offer (SDP) Answer (SDP)
Candidates
(SDP)
Media
Stream
Data
Channel
OnCandidate
AddCandidate
CreateAnswer
ICE
Bindings + tests
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
5. Signaling
server
Early 2015, peer-to-peer webRTC Model
OmniPotent PeerConnection
PeerConnection
JavaScript APP (Sender) JavaScript APP (Receiver)
PeerConnectionMedia
Stream
Data
Channel
ICE UADTLS
RTP
Code
c
CreateOffer
Gather
Transport
SetLocal
SetRemote
ICE UA DTLS
RTP
Code
c
Gather
Transport
SetLocal
SetRemote
Offer (SDP) Answer (SDP)
Candidates
(SDP)
Media
Stream
Data
Channel
OnCandidate
AddCandidate
CreateAnswer
ICE
Bindings + tests
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
7. Early 2015, peer-to-peer webRTC Model
Open Questions
1. How do I handle bandwidth preferences?
2. How do I check fingerprint (MITM attacks prevention)?
3. How do I handle codecs and their parameters?
=> Hum, ……. Just modify the SDP and pray it works.
4. How to know which pair of candidate is being used?
5. How do I restrict the ICE gathering (not to expose my IPs)
6. How do I get more info about ICE failures?
7. How do I make the (ICE) handshake faster?
8. How do I get rid of JSEP, SDP, O/A, …..?
9. Can I switch front / self view without renegotiation?
=> No can’t do.
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
8. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• Faster ICE
• Ports optimizations
• Simulcast
• Others in webrtc 1.0
9. Q2 2015, New W3C WebRTC charter
• => ORTC – WebRTC merge
• Notion of Object Model API (ORTC inspired)
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
http://ortc.org/wp-content/uploads/2014/08/ortc.html
10. Object Models?
• Each Object Does only one thing!
• Each Object can be probed for capabilities (JS)
• Parameters can be get/set dynamically (JS)
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
#8 (Edge)
http://ortc.org/wp-content/uploads/2014/08/ortc.html
11. Late 2015, Object Model API
Open Questions
Have a final API.
• How do I handle bandwidth preferences
• How do I check fingerprint (MITM attacks prevention)
• How to know which pair of candidate is being used?
• How do I get more info about ICE failures?
• How do I get rid of JSEP, SDP, O/A, …..?
• Can I switch front / self view without renegotiation?
Have some API, can do better or more.
• How do I handle codecs and their parameters
• How do I restrict the ICE gathering (not to expose my IPs)
Not addressed yet
• How do I make the (ICE) handshake faster?
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
12. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• Faster ICE
• Ports optimizations
• Simulcast
• Others in webrtc 1.0
13. WebRTC 1.0 = Object Model + JSEP
PLACEHOLDER FOR DRAWING
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
14. WebRTC 1.0 = Object Model + JSEP
Problems:
1. m-lines in SDP are intrinsically bi-directional
2. m-lines in SDP have attributes that fit neither
sender nor receiver object model (mid), or
that would be duplicated.
How to model the equivalent to a m-line on top
of the Object Model (so we can throw it later if
we decide to get rid of SDP, JSEP, ….) ?
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
16. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• Faster ICE
• Ports optimizations
• Simulcast
• Others in webrtc 1.0
17. What’s your flavor?
Full, Trickle, Lite
Feature flavor Full Trickle Lite
Make connectivity checks Yes Yes NO
Answer connectivity checks Yes Yes Yes
Controlling or controled Both Both Controlled
Candidates send in the Offer/Answer Yes No ?
Flag Ice-option:
trickle
a=ice-lite
https://tools.ietf.org/html/draft-ietf-mmusic-trickle-ice-02
https://webrtchacks.com/trickle-ice/
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
Trickle: From 10s to 1S
Q: How to get 100ms?
18. Faster ICE => more async, less waiting
- Start even before the call!
- ICE pool
- Pre-gathering
- Warmup:
- ICE, DTLS, media ready before signaling is.
- Early Media:
- Start sending media with answer
- Simplify & fasten the pairing / checking
- Default TURN server
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
19. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• Faster ICE
• Ports optimizations
• Simulcast
• Others in webrtc 1.0
20. RTP/RTCP within 1 PeerConnection
Use case: 2 media tracks (audio and video), how
many ports do I need? Less ports would useful
for
- Better connection rate esp. over mobile net.
- Optimize turn/media server resources
- Client side ports constraints
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
21. RTP/RTCP within a PeerConnection
Default
4
A
V
RTP RTCP
http://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation
http://datatracker.ietf.org/doc/rfc5761/
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
25. RTP/RTCP – extrapolation to n tracks
Default
2n
Bundle
n
Rtcp-mux
n
Both
1
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP
a=rtcp-muxa=group:BUNDLE
http://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation
http://datatracker.ietf.org/doc/rfc5761/
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
28. WebRTC / Object Model
Example 2: Audio + Video, 1:1
rtcp-mux
Rtp
Sender
Rtp
Receiver
Local
Track
NET
Remote
Track
Rtp
Sender
Local
Track
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Rtp
Transceiver
Rtp
Sender
Rtp
Receiver
Local
Track
NET
Remote
Track
Rtp
Sender
Local
Track
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Rtp
Transceiver
AUDIO
VIDEO
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
Rtcp-mux
n
30. Data
Channel
Data
Channel
WebRTC / Object Model
Generic & Default !
RtpTransceiver
RtpTransceiver
Rtp
Sender
Rtp
Sender
Rtp
Sender
Rtp
ReceiverTrack
Rtp
Sender
Rtp
Receiver
Dtls
Transport
Ice
Transport
Local
Track
NET Ice
Transport
Dtls
Transport
Track
Remote
Track
Rtp
ReceiverTrack
Remote
Track
Rtp
Sender Track
Local
Track
RtpTransceiver
RtpTransceiver
Sctp
Transport
Data
Channel
Sctp
Transport
Data
Channel
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
Both
1
31. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• ICE and Optimizations
• Simulcast
• Others in webrtc 1.0
32. Reminder
Multiparty vs Simulcast vs SVC
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
A
B
C
Multiparty
• Several tracks
• Decodable separately
• Bandwidth management separated
HD
SD
QVGA
Simulcast
• Several tracks
• Coming from the same source
• Decodable separately
• Smart bandwidth management possible
1SSRC
Δ2
Δ1
QVGA
SVC Encoding
• Several tracks
• Coming from the same source
• Not Decodable separately (Except base layer)
• Smart bandwidth management mandatory
• Less bandwidth, more resilience.
1SSRC
E
E
E
E
E
E
SVCEnc.
D
D
D
HD
SD
QVGA
33. Reminder
State of Multiparty signaling
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
From the beginning of webrtc, one have been able to
send multiple media streams in separate peer
connections, but it consumes much more ports, and
there is no sync between those tracks (lip sync, ..).
A
B
C
E
E
E
Multicast
• Several tracks
• Decodable separately
• Bandwidth management separated
34. Reminder
State of Multiparty signaling
For multiple streams in a single peer connection!
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
35. Reminder
Multiparty vs Simulcast vs SVC
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
A
B
C
Multicast
• Several tracks
• Decodable separately
• Bandwidth management separated
HD
SD
QVGA
Simulcast
• Several tracks
• Coming from the same source
• Decodable separately
• Smart bandwidth management possible
1SSRC
Δ2
Δ1
QVGA
SVC Encoding
• Several tracks
• Coming from the same source
• Not Decodable separately (Except base layer)
• Smart bandwidth management mandatory
• Less bandwidth, more resilience.
1SSRC
E
E
E
E
E
E
SVCEnc.
D
D
D
HD
SD
QVGA
36. Simulcast: Use case for webRTC 1.0
- Use Case for webrtc 1.0: SFU
- Browser send simulcast,
does not receive simulcast
(in webrtc 1.0)
Desk.
HD
SFU
SD
QVGA
Cr.
Book
Mobile
Desk.
HD
SD
QVGA
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
37. Simulcast (and SVC later):
3 layers, 3 working groups, 2 standard bodies
- Separate the
- “on-the-wire” info (1),
- JS API (2),
- and signaling (3)
Signaling
server
Web
server
Bob
(device)
Alice
(browser)
Media
(3)
(2)
(1)
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
38. Overview
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
• Early 2015 WebRTC
• Object Models APIs
• Glue (Tranceiver)
• ICE and Optimizations
• Simulcast
• Others in webrtc 1.0
39. Screen Capture API simplification
- No more distinction between screen, windows
and apps. => no more constraints, flat list.
• Screen Capture
http://w3c.github.io/mediacapture-screen-share/
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
40. Record your streams (local), but not all streams (remote).
http://www.w3.org/TR/mediastream-recording/
Media Stream Recording (Client side)
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
41. Use <AUDIO> or <VIDEO> as a source !
http://w3c.github.io/mediacapture-fromelement/
Media Capture from DOM Elements
&& Image Capture (a.k.a. snapshot)
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
42. Send different streams to different outputs
http://w3c.github.io/mediacapture-output/
Audio output devices API
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0
45. SECURITY: IP Address Leakage
4 levels for candidate gathering
- All [with consent]
- Restricted I (host + 1918) [default]
- Restricted II (no host) [pref/extension]
- Only proxy/VPN [pref/extension]
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
46. Conclusion
WebRTC 1.0 is Awesome [Evil Grin]
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0
Editor's Notes Note alex, add arrows on the right side