Description of CoSMo and its solution for client-side support of Janus servers and other webrtc media servers and services.

1. Janus Client-Side
Dr Alex Gouaillard
CoSMo Software

2. WARNING !
2
● This is a pineapple-free presentation. No pineapple has been used during
the making of this presentation, and no pizza was tortured nor coerced to
have an indecent relationship with a pineapple!

3. WARNING !
3
● This is a pineapple-free presentation. No pineapple has been used during
the making of this presentation, and no pizza was tortured nor coerced to
have an indecent relationship with a pineapple!
● This is a VoIP-free presentation! The words Kamailio, FreeSwitch, and
Asterisk will have no part in this presentation. If this hurts your feelings, or
your (religious) beliefs, …. C’est la vie!

4. We are CoSMo
We used to cure cancer with ML and AI (ten years ago),and then webrtc came along.
4

5. We are CoSMo
● Based in Singapour (HQ), Manila, France, Madrid and L.A. (streaming)
● Multiple open-source collaborators
○ Google,
○ Apple,
○ Xyrsis,
○ Meteecho,
○ ...
● Gift economy:Contributions
to standards and testing
○ W3C
○ IETF
○ IMTC
○ AOMedia
5

6. We Love (C++) WebRTC, C, ASM, codecs, and testing
Which makes us very popular because nobody else do.
● Contribution to Open-Source
○ KITE (With Google, Used by Apple and others)
○ No-code fuzzing
○ E2EME (PERC, SFrame)
○ H264 simulcast
● Ongoing Open-Source
○ H.265 HW accelerated (with INTEL and Apple) => working, being tested by A, prep for upstream
○ Real-time AV1 => Proposed to AOM, to be presented at board meeting sep 25th
○ Media Transport / Web Transport / Datagram on QUIC => working impl. On top of picoQuic
● Non-open source research
○ Narval
○ Real-Time Server Side Ad-Insertion
○ Real-Time Digital Forensic Watermaking of Video Media
6

7. We are CoSMo
● Free Testing results and white papers for the community at large
● IETF HAckathons open to all (including commercial non open-source media
servers).
7

8. W3C TPAC 2019 - WPT.kite
8

9. W3C TPAC 2019 - WPT.kite
9

10. We have our own Media Server ...
10

11. We still love Janus!
● Video Conferencing vs Streaming
● Hybrid ? (webinar, conference, courses, …..)
● IoT? => 100% Janus
● Legacy?
○ VoIP => 100% Janus
○ Streaming (RTMP, HLS, MPEG-DASH, ….)
The fact that the Janus team participate to IETF, is so knowledgeable, and
answers so fast makes the difference for us.
11

12. Client-side, server-side, PaaS and operation
Need to Collaborate
12

13. The WebRTC A-Team(s)
10 years ago, a crack student unit was sent to PhD courses by a European court
for WebRTC, a technology they didn’t write. These men promptly graduated from
maximum security universities to the Napoli and Singapore Underground.
Today, still wanted by their government-paid past teachers, they survive as
WebRTC consultants.
If you have a WebRTC problem, if no one else can help, and if you can find them,
maybe you can hire… the WebRTC A-Team.
13

14. The WebRTC A-Team(s)
14

15. 15
Note:
Only real Rockstar:
Lorenzo Miniero

16. There is more to webrtc than p2p
There is more to webrtc than media server
0. Minimum viable SYSTEM:
● 1 Client app (web app in chrome)
● 1 server
1. Real Media Infrastructure:
● Load balancing
● Auto-scaling
● As many client types as possible
● Metrics
● ICE servers
2. Platform as a Service:
● Authentification
● Accounts, Tenants management
● Payment
● Recording, Ad, Watermarking, ...
ANNEX: Tooling:
● Unit testing
● Interop testing
● CI/CL
● Load Testing 16

17. There is more to webrtc than p2p
There is more to webrtc than media server
0. Minimum viable SYSTEM:
● 1 Client app (web app in chrome)
● 1 server
1. Real Media Infrastructure:
● Load balancing
● Auto-scaling
● As many client types as possible
● Metrics
● ICE servers
2. Platform as a Service:
● Authentification
● Accounts, Tenants management
● Payment
● Recording, Ad, Watermarking, ...
ANNEX: Tooling:
● Unit testing
● Interop testing
● CI/CL
● Load Testing 17

18. Let’s dive! Libwebrtc is easy
You want to add a video codec?
Just slide it in the encoder part of the code.
Easy right … ?
18

19. Let’s dive! Libwebrtc is easy
Well, wait,
You need the decoder too,
And the rtcp feedback loop …. 19

20. Let’s dive! Libwebrtc is easy
+ JSEP, + SRTP, +ICE, ...
20

21. As a Product maintainer …
I do not want to deal with fetching, building, or
validating libWebRTC!
I want to link against external / system libWebRTC
○ More Options
■ Additional Codecs, E2EE, OpenSSL instead of BoringSSL
○ pre-built
■ Architectures (x86_64, arm64), compilers
■ Release only, No source code, Artefact Factory
○ validated
■ Google Unit Tests, End-to-end testing, Dashboard
○ packaged
■ relocatable, Installable, Usable without DevEnv
○ “Importable” in other DevEnvs
■ CMake package, Toolchains (android!)
I want to rebase / update easily
○ Commit and branch exact Versioning of binaries
○ Migration guides / scripts
21

22. Unit Testing 1/4
22

23. Unit Testing 2/4
23

24. Unit Testing 3/4
24

25. Unit Testing 4/4
25

26. CoSMo Add-On(s) to Janus Janus
Server
SFU
Qt Desk. &
Mobile
Client
Electron
Desktop
Client
Signal
lib
Web
RTC
lib
Web
RTCFull
Custom
+
E2E
Encryp
lib
Web
RTC
OBS-Studio
Signal
lib
Web
RTC
Signal
WebRTC – Qt
Wrappers
26

27. 27

28. Java/ obj-c
Wrapper
C++ API
arm
C++ API
x64
libWebRTC Build Types
custom build options
E2EE
OpenSSL 1.1g
Watermarking
Additional Codecs
G.729
VP9 SVC
Medooze
Jitsi
Janus
Web App
React Native
App
(iOS/android)
OBS-Studio
Qt App
Desktop
M
obile
&
IoT
IoT
Electron
App
Browsers
Chrome
Firefox
Safari
Edgium
Native
Mobile
Java API
C++ API
obj-c API
Swift API
Millicast
Raspberry Pi
TizenOS
Native Flutter
Mobile App
69
28

29. People come at it from all different angles: Web First
1. Most people Starts with a web app in Chrome
2. Extending to more browser is already
Challenge in itself even if it’s getting better
(see next slide)
3. Then people want native, desktop or mobile,
With minimum extra effort. It means reusing
JS code (signalling, UI/UX, logic) The usual suspects:
● React-native (mobile) => needs libwebrtc
● Electron (Desktop)
29

30. 2018 ~ 2019 - WPT.fyi: many more passing tests
30

31. People come at it from all different angles: Native First
1. Many people are mobile only
2. Most of those use libwebrtc for the media
Stack. Compilation can be tricky.
3. Then people want native, desktop or mobile,
With minimum extra effort. It means reusing
Native code (signalling, UI/UX, logic).
No usual suspect outside of full default apps
31

32. We had a dream !
1. Start with web app for simplicity
2. Reuse JS signalling APi and logic in native app to reduce code duplication
3. Except for JS, No binding (java, obj-C, ….),
4. Single source code (ASM/C/C++) for mobile and native !
5. Professional cross-platform UI framework
6. Professional cross-platform Build/Test/Report tools
And someone gave us QT (+ cmake)!
32

33. Dreams do come true!
It just takes more sweat than you think.
1. Build system, QtCreator
2. Thread model,
3. Memory Model,
4. Garbage Collection,
5. JS reuse,
a. QML auto-binding
b. Strings / QStrings
6. Capturer (Desktop/App capture)
7. Rendering (Image Adaptation, OnFrame callback)
8. Websocket (QtWebsocket subprotocol support limitation)
33

34. 34