Presentation of my last talk at MilanoJS event. It is the case history of a project where we used the WebRTC to make a chat skype style, using also WebSockets with Socket.io on NodeJS

1. WEBRTC + SOCKET.IO
BUILDING A SKYPE-LIKE VIDEO CHAT WITH
NATIVE JAVASCRIPT
/MICHELE DI SALVATORE @MIKDISAL
Javascript Architect @ Objectway

2. Audio and video communication and peer-to-peer data
sharing through a web application
Native javascript (no plugins)
Open source
appear.in

3. HOW IS BORN
Global IP Solutions acquired by Google in 2010
Google open sourced the technologies

4. SUPPORTED BROWSERS AND PLATFORMS

5. IE AND SAFARI?
freeTemasys Plugin

6. JAVASCRIPT WEBRTC API
MediaStream
RTCPeerConnection
RTCDataChannel

7. MEDIASTREAM (AKA GETUSERMEDIA)
Acquiring audio and video
Can contain multiple 'tracks'
var constraints = {
video: {
mandatory: {
minWidth: 640,
minHeight: 360
}
};
function successCallback(stream) {
var video = document.querySelector("video");
video.src = window.URL.createObjectURL(stream);
}
function errorCallback(error) {
console.log("navigator.getUserMedia error: ", error);
}
Constraints controls the contents of the MediaStream

8. RTCPEERCONNECTION
Communicating audio and video
Signal processing
Codec handling
Peer to peer communication
Security
Bandwidth management
...

9. RTCDATACHANNEL
Communicating arbitrary data
Same API as WebSockets
Ultra-low latency
Unreliable or reliable
Secure

10. SIGNALING
Exchanging 'session description' objects
What formats I support, what I want to send
Network information for peer-to-peer setup
Any message mechanism and protocol

11. JSEP
JavaScript Session Establishment Protocol

12. RTCSESSIONDESCRIPTION EXAMPLE
v=0
o=- 7614219274584779017 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:W2TGCZw2NZHuwlnf
a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=mid:audio
a=rtcp-mux
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiC
a=rtpmap:111 opus/48000/2
...

13. P2P IN REAL WORLD
NAT & Firewalls?

14. STUN SERVER
Gives my public IP address
Simple server, cheap to run
Data flows peer-to-peer

15. STUN SERVER

16. TURN SERVER
Fallback for p2p failing
Data via server
Call works everywhere

17. STUN + TURN SERVERS

18. ICE
Interactive Connectivity Establishment
A framework to connect peers via UDP, enabling
RTCPeerConnection via STUN and TURN server

19. WEBRTC SECURITY
On Chrome only via https
Encryption for media and data
Sandboxed

20. WEBRTC LIBRARIES & CO.
by
by
by
by
...
adapter.js webrtc
rtc-everywhere contra
SimpleWebRTC &Yet
RTCMultiConnection Muaz Khan

21. RTCMULTICONNECTION
A javascript wrapper library supporting
approximately all possible peer-to-peer
features.
WEBRTC EXPERIMENTS
Tons of open source experiments based on
RTCMultiConnection:
webrtc-experiment.com

22. WEBSOCKETS
Standardized in 2011
Interactive communication session between browser and
server
Event driven
Sending strings and binary data

23. WEBSOCKETS BROWSER SUPPORT
All and IE10+
ajax polling as fallback

24. SOCKET.IO
A popular open source library with client and server
implementations
Server side written in Node and easy to start
var server = require('http').createServer();
var io = require('socket.io')(server);
io.on('connection', function(socket){
socket.on('hi!', function(data){
socket.emit('a-message', {hello: 'world'});
});
socket.on('disconnect', function(){});
});
server.listen(3000);

25. SOCKET.IO CLIENT
var socket = io.connect('http://localhost:3000/');
socket.on('connect', function () {
socket.emit('hi!');
});
socket.on('a-message', function (data) {
console.log(data.hello);
});

26. GOAL
A SKYPE LIKE CHAT

27. MAIN ISSUES
WebRTC architecture like IRC
rooms with shared chat: text, video and data
direct chat: only one to one
room managed by initiator
conversation type can change without agreement: text to
video

28. IMPLEMENTATION

29. STEP 1
UserA is logged in and is connected to the socket server

30. STEP 2
The app sends a "presence" event to other users through
the socket
The server forwards the message only to user's contacts
Each present user replies with the same "presence" event

31. SOCKET.IO FORWARD ROLE

32. STEP 3
Each present user replies with the same "presence" event

33. STEP 4
When a user receives a "presence" event, it automatically
knows that the sender is online
Also the server sends the “presence” event to all, when a
user socket is disconnected

34. STEP 5
UserA starts a conversation and the app creates the webrtc
connection

35. STEP 6
UserA sends a message via webrtc

36. STEP 7
UserB receives a notification

37. STEP 8
UserB starts writing a message

38. STEP 9
UserA receives the "is typing" info sent via webrtc

39. STEP 10
UserA starts a video call sending a request to UserB via
socket

40. STEP 11
UserB receives the request and accepting starts a video call
via webrtc

41. STEP 12
The 2 users can video chat via webrtc
Each user can close the video at any time or start a text chat

42. STEP 13
A user can send text messages also

43. STEP 14
A user can start an audio chat in the same way as a video
chat

44. chat
BEHIND THE SCENES

45. WHERE IS WEBRTC?
Behind this actions
chat messages
video / audio call
"is typing" info

46. AND THE OTHERS?
via WebSockets
presence
video / audio initialization
all status infos

47. THAT'S ALL!
THANK YOU ALL
/Michele Di Salvatore @MikDiSal