FOSDEM 2016 - Creating rich WebRTC Applications with Kurento
•Download as PPTX, PDF•
5 likes•2,160 views
This document discusses Kurento, an open-source WebRTC infrastructure and its APIs. Kurento provides a media server and client APIs that allow developers to create rich multimedia applications. The document outlines Kurento's modular client API, media processing capabilities like recording and computer vision, and higher-level APIs for group communications. It also describes how to develop custom media processing modules and provides resources for getting started with Kurento.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to FOSDEM 2016 - Creating rich WebRTC Applications with Kurento
Similar to FOSDEM 2016 - Creating rich WebRTC Applications with Kurento (20)
Recently uploaded
Recently uploaded (20)
FOSDEM 2016 - Creating rich WebRTC Applications with Kurento
- 1. Creating rich WebRTC Applications with Kurento Luis Lopez lulop@kurento.org
- 2. Who I am http://www.kurento.org - lulop@kurento.org 2 Associate Professor Escuela Técnica Superior de Ingenieros de Telecomunicación Universidad Rey Juan Carlos (south Madrid) Researcher in the area of RTC • http://www.nubomedia.eu • http://www.fiware.org FOSS enthusiast • Kurento project lead: http://www.kurento.org • NUBOMEDIA Community lead: http://www.nubomedia.eu
- 3. Real-Time media Communications http://www.kurento.org - lulop@kurento.org 3 Capture Encode Cipher Transport Transport Decipher Decode RenderNetwork The media plane The signaling plane I wan to call you, do you accept? Yes, do it in this way
- 4. WebRTC to the rescue. Before WebRTC First wave of WebRTC technologies Begin End • APIs • Standards • FOSS Development experience when working with real-time media Common WebRTC application (p2p communications) WebRTC video stream http://www.kurento.org - lulop@kurento.org 4
- 5. WebRTC infrastructures http://www.kurento.org - lulop@kurento.org 5 Peer-to-Peer WebRTC Application (without media infrastructure) WebRTC video stream WebRTC Application based on media infrastructure media infrastructure
- 6. WWW VS (Web)RTC applications http://www.kurento.org - lulop@kurento.org 6 Control Multimedia Application logic (developers’ code) Multimedia Capabilities Recording Transcoding Routing Mixing Analyzing Etc. Media Traffic RTC Media APIs Multimedia Clients Events Application Signaling Control WWW Application logic (developers’ code) Database Capabilities Data storage Data recovery Data querying Data processing Etc. DD.BB. API WWW Clients Events Application Signaling
- 8. http://www.kurento.org - lulop@kurento.org 8 Control Multimedia Application logic (developers’ code) Multimedia Capabilities Recording Transcoding Routing Mixing Analyzing Etc. RTC Media APIs Events
- 9. Kurento: a WebRTC infrastructure an its APIs http://www.kurento.org - lulop@kurento.org 9 Control Multimedia Application logic (developers’ code) Recording Transcoding Routing Mixing Analyzing Adapting Media Traffic RTC Media APIs Multimedia Clients Events Application Signaling Computer vision Augmented reality Blending Etc. Multimedia Capabilities Kurento Media Server (KMS) Kurento Client API Kurento Room API Kurento Tree API
- 11. The Kurento FOSS Community http://www.kurento.org - lulop@kurento.org 11 Community support WebRTC worldwide reference International awards More than 300 companies Very active mailing list First result in Google
- 12. The Kurento FOSS project http://www.kurento.org - lulop@kurento.org 12 • http://www.kurento.org – Main web site • https://www.twitter.com/kurentoms – Main social channel • https://groups.google.com/forum/#!forum/kurento – Main mailing list • https://github.com/kurento – Main repository • https://www.youtube.com/channel/UCFtGhWYqahVlzMgGNtEmKug – Kurento Youtube channel
- 13. Developing with KMS: The Kurento Client API http://www.kurento.org - lulop@kurento.org 13 SinkSRC Sink SRC SRCSink Sink Media Element • Provides a specific media functionality › Send/receive media › Process media › Transform media • Exchange media through › Sources › Sinks Media pipeline • Chain of media elements implementing the desired media logic. • The Media API provides the capability of creating media pipelines by joining media elements of the toolbox Media Element Sink SRC
- 14. A modular API • Modular in the sense of modularity* – Isolation • Internal states of a module don’t affect the rest – Abstracion • Internal states are hidden behind an interface – Composability • Interface must enable module recombination and assembly – Reusability • If a module has it, you can use it – Extensibility • If a module does not have it, you can add it. http://www.kurento.org - lulop@kurento.org 14 * Baldwin, Carliss Young, and Kim B. Clark. Design rules: The power of modularity. Vol. 1. MIT press, 2000
- 15. The paradigm of modularity http://www.kurento.org - lulop@kurento.org 15
- 16. The Kurento Client API in one word http://www.kurento.org - lulop@kurento.org 16 sourceElement.connect(sinkElement) connect
- 17. http://www.kurento.org - lulop@kurento.org 17 Kurento hello world
- 18. //MediaPipeline is a holder of media elements MediaPipeline pipeline = kurento.createMediaPipeline(); //Create your media elements WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); //Connect your media elements webRtcEndpoint.connect(webRtcEndpoint); http://www.kurento.org - lulop@kurento.org 18 Control Multimedia Application logic (developers’ code) Recording Transcoding Routing Mixing Analyzing Adapting Kurento Media APIs Multimedia Clients Events Application Signaling Computer vision Augmented reality Blending Etc. Multimedia Capabilities //Where to render the local and remote streams var options = { localVideo : videoInput, remoteVideo : videoOutput, onicecandidate : onIceCandidate } //Start media capture and communications webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv( options, function(error) { if (error) return console.error(error); webRtcPeer.generateOffer(onOffer); }); Application Server Code (Java) Client Server Code Application source for Spring (Java EE) Application source for Node.js (JavaScript)
- 19. Recording http://www.kurento.org - lulop@kurento.org SinkSRC Sink //MediaPipeline is a holder of media elements MediaPipeline pipeline = kurento.createMediaPipeline(); //Create your media elements WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); RecorderEndpoint recorderEndpoint = new RecorderEndpoint.Builder(pipeline).build( “file:///myfolder/myfile.mp4”); //Connect your media elements webRtcEndpoint.connect(webRtcEndpoint); webRtcEndpoint.connect(recorderEndpoint); 19 Application source for Spring (Java EE) Application source for Node.js (JavaScript)
- 20. Playing http://www.kurento.org - lulop@kurento.org 20 SinkSRC SRC //MediaPipeline is a holder of media elements MediaPipeline pipeline = kurento.createMediaPipeline(); //Create your media elements WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); PlayerEndpoint playerEndpoint = new PlayerEndpoint.Buildr(pipeline).build( “file:///myfolder/myfile.mp4”); //RSTP and HTTP URIs also supported //Connect your media elements playerEndpoint.connect(webRtcEndpoint); Application source for Spring (Java EE) Application source for Node.js (JavaScript)
- 21. Interoperating http://www.kurento.org - lulop@kurento.org 21 Media Pipeline WebRTC audio streaming SinkSRC SRCSink Media Pipeline SinkSRC SRCSink WebRTC video streaming RTP audio streaming RTP video streaming Transparent transcoding!!
- 22. Some Media processing modules • Augmented reality – MarkerArModule – MarkerlessArModule • Computer vision – FaceDetector – NoseDetector – EyeDetector – CrowdDetector – PointerDetector – MotionDetector – VirtualFence – Etc. http://www.kurento.org - lulop@kurento.org 22 Sink SRC Sink SRC Sink SRC Sink SRC Sink SRC
- 23. Using media processing modules http://www.kurento.org - lulop@kurento.org 23 SinkSRC Sink SRC CrowdEvents SRC Sink SRC CrowdDetector RtspPlayer SinkSRC See demo See demo
- 24. Implementing group communications: just connect http://www.kurento.org - lulop@kurento.org 24 Media Pipeline WebRTC streaming SinkSRC WebRTC streaming SRCSinkSRCSink WebRTC streaming WebRtcEndpoint 1 WebRtcEndpoint 3 WebRtcEndpoint 2 SRCSink WebRTC streaming WebRtcEndpoint 4 User 1 User 2 User 3 User 4
- 25. Group communications: RTP topologies http://www.kurento.org - lulop@kurento.org 25 SinkSRC SRC Sink SinkSRC SRCSink • Media Mixing Mixer – Composite • Media Switching Mixer – Connect primitive – Natural topology • SFU – WebRtcSfu • Simulcast
- 26. Creating your own modules http://www.kurento.org - lulop@kurento.org 26 Kurento IDL Compiler JSON Module IDL Description Java Java- Script cpphpp Media API code Toolchain Auto-generated code Module Implementation cmake Interface Developer (1) (2) (3)
- 27. Going up the API stack: the room API http://www.kurento.org - lulop@kurento.org 27 createRoom deleteRoom joinRoom leaveRoom publisMedia unpublishMedia onParticipantJoined onParticipantLeft onMediaPublished onMediaUnpublished onMessage See demo
- 28. Going up the API stack: the tree API http://www.kurento.org - lulop@kurento.org 28 createTree deleteTree joinAsPresenter joinAsViewer leave
- 29. What’s next http://www.kurento.org - lulop@kurento.org 29
- 30. Where to start http://www.kurento.org - lulop@kurento.org 30 • http://www.kurento.org/documentation – Installation and administration guide – Java tutorials for Spring and Java EE developers – JavaScript tutorials for Node.js developers – JSON-RPC protocol documentation • https://github.com/kurento – Main repository
- 31. Thanks http://www.kurento.org - lulop@kurento.org 31 Luis Lopez lulop@kurento.org