This document introduces MPEG-M (MPEG Extensible Middleware), which provides a standardized set of APIs to access various MPEG technologies through "engines". The MXM APIs simplify application development by hiding the complexity of MPEG standards. The MXM architecture includes engines for MPEG-21, MPEG-4, IPMP, and other technologies. Applications can access these engines through simple MXM API calls to perform tasks like streaming media, including 3D graphics, and sharing protected content. The goal of MXM is to accelerate media business by providing a common platform and set of protocols for distributed multimedia applications and services.

1. http://mxm.wg11.sc29.org/ mxm@lists.uni-klu.ac.at http://wg11.sc29.org/mxmsvn/repos Accelerating Media Business Developments MPEG-M: MPEG Extensible Middleware Christian Timmerer, FilippoChiariglione, Marius Preda Klagenfurt University (UNI-KLU)  Faculty of Technical Sciences (TEWI) Department of Information Technology (ITEC)  Multimedia Communication (MMC) http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.at Acknowledgments L. Chiariglione, M. Eberhard, I. Arsov, A. Difino

3. … one is able to start with application/business development as soon as some (reference) software becomes available?

4. … one is able to exchange applications’ underlying (reference) software with optimized one at no cost?2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 2

5. 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 3

6. Outline Vision Overview Architecture Application Programming Interface (API) Example Instantiations Fully Interoperable Streaming Including MPEG-4 3D Graphics in 3rd-Party Apps Sharing Protected Contents 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 4

8. From Framework to Platform respecting

9. Creator and rights holders rights to exploit their works

10. End user wish to fully enjoy the benefits of digital media

11. Various value-chain player interest to provide products and services➪ DMP has specified Interoperable DRM Platform (IDP) 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 5 “every human is potentially an element of a network involving billions of content providers, value adders, packagers, service providers, resellers, consumers ...” Framework Platform . . .

14. ITU-T: definition of IPTV infrastructure and components

15. MPEG: development of enabling technologies for, e.g., IPTV➪ Advanced IPTV Terminal (AIT) 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 6 . . . Middleware Terminal

17. Simple methods to call complex functionalities inside MXM engines

18. “Thin” applications because the complexity is in the MXM engines

19. Replacement of MXM engines with better performing ones at no cost

20. Creation of a global market of MXM Engines, MXM Applications and MXM Devices2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 7

22. Part 2 - MXM Application Programming Interfaces (APIs): specifies the MXM APIs;

23. Part 3 - MXM Conformance and Reference Software: specifies conformance tests and the software implementation of the standard

24. Part 4 – MXM Protocols: specifies a set of protocols enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection information2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 8

25. MXM Application 1 MXM Application 2 MXM Engine APIs‏ MXM Device MPEG21 File Engine Digital Item Engine REL Engine IPMP Engine Other Engines Security Engine Scene Engine Content Metadata Engine Media Framework Engine OS Drivers, Accelerators, Controllers, etc. HW 9 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

26. MXM Application 3 MXM Application 1 MXM Application 2 MXM Orchestrator API‏ MXM Device MPEG21 File Engine Digital Item Engine REL Engine IPMP Engine Orchestrator Engine Other Engines Security Engine Scene Engine Content Metadata Engine Media Framework Engine OS Drivers, Accelerators, Controllers, etc. HW 10 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

27. MXM Protocols License Provider Device Content Identific. Device Content Creation Device End-User Device Content Provider Device DRM Tool Provider Device 11 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

28. MXM Application MXM Application MXM OS MXM OS Computing Platform Computing Platform Enabling MXM Apps to communicate 12 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

29. An Impressive Number of Standards… MPEG-7 Visual Advanced Audio Coding LASeR MDS Event Reporting Reconfigurable Video Coding BIFS HE AAC IPMPX Digital Item Processing Digital Item Declaration File Format Audio Lossless Coding Intellectual Property Management and Protection Media Value Chain Ontology Rights Expression Language Digital Item Identification XML IPMP messages Digital Item Streaming IPMP Components Digital Item Adaptation MPQF 13 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

30. Motivation of providing API MPEG specifications: huge amount of technology 157 standards in ISO/IEC 14496 (MPEG-4) family 43 standards in ISO/IEC 15938 (MPEG-7) family 33 standards in ISO/IEC 21000 (MPEG-21) family … Around 11 000 pages, 1,5 m high when printed However, majority of people developing MPEG related applications do not need to know that is inside the boxes, but only how to use it 2009/11/23-24 14 Christian Timmerer, Klagenfurt University, Austria

31. The MXM Approach 1. Creating wrapping libraries, called engines 2. Opening the engines at ESSENTIAL points only 3. Documenting the IN/OUT points in another … MPEG standard: 23006-2 MXM API - only 37 pages , - around 500 methods 2009/11/23-24 15 Christian Timmerer, Klagenfurt University, Austria

33. ContentSearchEngine

34. DIAEngine

35. DIDEngine

36. DISEngine

37. DomainEngine

38. EREngine

39. IPMPEngine

40. IPMPToolProtocolEngine

41. LicenseProtocolEngine

42. MediaFrameworkEngine

43. MetadataEngine

44. MPEG21FileEngine

45. MVCOEngine

46. OrchestratorEngine

47. RELEngine

48. RenderingEngine

49. SecurityEngine16 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

51. Encode a raw audio track

52. Create an MPEG-7 metadata description

53. Access e.g.

54. Get data from a Digital Item

55. Decode a video

56. Editing e.g.

57. Add an elementary stream to a multiplexed content

58. Engine-specific e.g.

59. RELEngine: authorise(license, query)

60. LicenseProtocolEngine: requestLicense(licenseID, serviceURL) 17 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

61. Fully Interoperable Streaming ofMedia Resources in Heterogeneous Environments 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 18

62. Including MPEG-4 3D graphics in 3rd-Party Application Including mp3, jpeg, mp4 video in third party applications is nowadays a beginner job. The complexity of such codecs is hidden behind a very simple communication interface once the content is decoded: matrix of pixels for images and wave samples for audio. Transposing the same principle in Computer Graphics world is a challenge due to the variety of representation forms and also the complexity and heterogeneity of data to be transferred 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 19 By using the MXM 3DGraphicsEngine and its set of APIs, the complex integration work is simplified. With only some lines of code, Ogre3D, a very well known 3D graphics rendering engine, is transformed into an MPEG-4 3D graphics player.

63. Sharing Protected Content 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria 20 Client: MXM-based C++ client as Firefox plug-in Server: MXM-based Java server Protocols: MXM Protocols over SOAP and XMPP http://www.smartrm.com

64. Join the MXM Development Team! Why you should join Plenty of interesting work still has to be done Enough space for student projects, master thesis, PhD thesis, etc. You can choose between Java, C++ or start a new implementation in another language! It gives you visibility on a broad set of MPEG technologies Web site, blog, reflector, and soon more utilities Friendly and collaborative environment  21 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria

65. Thank you for your attention! ... questions, comments, etc. are welcome … Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer Klagenfurt University, Department of Information Technology (ITEC) Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA christian.timmerer@itec.uni-klu.ac.at http://research.timmerer.com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Christian Timmerer 22 2009/11/23-24 Christian Timmerer, Klagenfurt University, Austria http://mxm.wg11.sc29.org/ mxm@lists.uni-klu.ac.at http://wg11.sc29.org/mxmsvn/repos