Behind the scenes of the CMAF standard supported by Apple and Microsoft.

1. Institut Mines-Télécom
CMAF 101
Cyril Concolato
Paris Video Tech Meetup #3
01/02/2017
GPAC

2. Institut Mines-Télécom
Disclaimer
These are my views/understanding
(based on Jan. 2017 version),
not necessarily the views of the
CMAF group or of the other chairs
or the final standard
Cyril Concolato - Paris Video Tech #32

3. Institut Mines-Télécom
Adaptive Streaming Market Fragmentation
Cyril Concolato - Paris Video Tech #33

4. Institut Mines-Télécom
CMAF
 Specification mainly proposed by
• Apple
• Microsoft
 Defines a Format for Segmented Delivery
• To improve cacheability, to reduce CDN costs
• To reduce deployment complexity, to remove the need
for gateways/transformatters
Cyril Concolato - Paris Video Tech #34

5. Institut Mines-Télécom
C-MAF: What is a MAF ?
 MPEG usually makes wide, general-purpose,
application-agnostic formats
• Ex: Codecs (AVC, HEVC …) can be used for
─ Broadcast, on-demand, video conferencing systems …
• Ex: Containers can be used in many application types
─ MPEG-2 TS is used in DVB, ATSC, HLS, …
─ MP4 is used as camera files, web streams …
 MAF = MPEG Application Format
(ISO/IEC 23000, a.k.a. MPEG-A)
• Interesting combinations of MPEG standards for
dedicated applications
Cyril Concolato - Paris Video Tech #35

6. Institut Mines-Télécom
Lots of MAFs
ISO/IEC 23000-2: Music player application format
ISO/IEC 23000-3: Photo player application format
ISO/IEC 23000-4: Musical slide show application format
ISO/IEC 23000-5: Media streaming application format
ISO/IEC 23000-6: Professional archival application format
ISO/IEC 23000-7: Open release access application format
ISO/IEC 23000-8: Portable video player application format
ISO/IEC 23000-9: Digital Multimedia Broadcasting application format
ISO/IEC 23000-10: Video surveillance application format
ISO/IEC 23000-11: Stereoscopic video application format
ISO/IEC 23000-12: Interactive music application format
ISO/IEC 23000-13: Augmented reality application format
ISO/IEC 23000-15: Multimedia preservation application format
ISO/IEC 23000-16: Publish/Subscribe application format
ISO/IEC 23000-17: Multiple Sensorial Media application format
ISO/IEC 23000-18: Media Linking application format
ISO/IEC 23000-19: Common media application format
ISO/IEC 23000-20: Omni-directional Media application format
Cyril Concolato - Paris Video Tech #36

7. Institut Mines-Télécom
CMAF Development Timeline
A very fast standardization pace
Feb. 2016
Working
Draft
June 2016
Committee
Draft
Nov. 2016
Draft
International
Standard
Jan. 2017
Study of
Draft
International
Standard
Apr. 2017
Final Draft
International
Standard
July 2017
Approved
International
Standard
Cyril Concolato - Paris Video Tech #37

8. Institut Mines-Télécom
The CMAF Specification
 Covered
• Media Format: choice of MP4 / CENC & constraints
• Media Profiles: choice of codecs (AAC, AVC, HEVC) &
constraints
• Presentation Profiles: set of Media Profiles
 Not covered
• No new ISOBMFF features, no new boxes
─ Content should play in existing players
• No new manifest format
• No specific player behavior
 Not completed
• Encryption modes: CBC vs. CTR
─ CMAF only fully useful today for non-protected content
Cyril Concolato - Paris Video Tech #38

9. Institut Mines-Télécom
First time reading the CMAF spec:
Why is it so complex?
https://www.youtube.com/watch?v=oLxFqzumFrc
Cyril Concolato - Paris Video Tech #39

10. Institut Mines-Télécom
Basic Concept: Segmented Delivery
 Ability to process parts of a stream independently
• At the encoding level: open/closed GoP
• At the encryption level: encryption metadata
• At the packaging level: self-contained ISOBMFF data
 Ability to switch between streams without double-
download and double-decode
• At the encoding level: time-alignment of closed GoPs
• At the encryption level: shared encryption parameters
• At the packaging level: shared ISOBMFF parameters
Cyril Concolato - Paris Video Tech #310
=> CMAF Fragment

11. Institut Mines-Télécom
Basic Concept: CMAF Fragment
 A set of constraints to guarantee
• Independent parsing
• Independent decryption
• Independent decoding
 Additional mappings to delivery units for
• On-Demand Delivery
• Live Delivery
• Low Latency Delivery
 Remember:
A CMAF Fragment is not a delivery concept!!!
Cyril Concolato - Paris Video Tech #311

12. Institut Mines-Télécom
Basic Concept: CMAF Track
 Set of contiguous CMAF Fragments
• single media type (no multiplex)
 Associated to a CMAF Header
 Remember:
A CMAF Track is not a delivery concept!!!
Cyril Concolato - Paris Video Tech #312

13. Institut Mines-Télécom
Delivering CMAF Fragments:
Hypothetical ways of forming delivery units
Cyril Concolato - Paris Video Tech #313
Encoder
Encryption
CMAF
Header
CMAF Fragments
On-demand
Packaging
CMAF
Track File
All
[0-duration]
1
Live
Packaging
CMAF
Segment1
or
more
1
CMAF Track
Low Latency
Packaging
1 N
CMAF Chunks

14. Institut Mines-Télécom
Real CMAF Chunk Production
Cyril Concolato - Paris Video Tech #314
Encoder
Encryption
CMAF
Header
Low Latency
Packaging
CMAF Chunks

15. Institut Mines-Télécom
CMAF Resources
 Manifests will typically provide URLs to
• CMAF Track Files
• Or CMAF Header + CMAF Segments
• Or CMAF Header + CMAF Chunks
 CMAF Resource Signaling via ISOBMFF brands
• ‘cmfc’ (general CMAF compatibility)
• ‘cmfs’ (CMAF segment compatibility)
• ‘cmfl’ (CMAF Chunks compatibility)
Cyril Concolato - Paris Video Tech #315

16. Institut Mines-Télécom
CMAF Chunks
CMAF
Header
CMAF Content consumption
Cyril Concolato - Paris Video Tech #316
Un-
packaging
MSE
CMAF
Track File
CMAF Fragments
Switching can only happen seamlessly
at CMAF Fragment boundaries
CMAF
Segments
CMAF
Header
CMAF
Header

17. Institut Mines-Télécom
Switching with decoder reinitialization
Cyril Concolato - Paris Video Tech #317
4321
CMAF Fragments
Quality 1
4321
CMAF Fragments
Quality 2
CMAF
Header
CMAF
Header
21
CMAF
Header
43
CMAF
Header

18. Institut Mines-Télécom
Switching without decoder reinitialization
Cyril Concolato - Paris Video Tech #318
4321
CMAF Fragments
Quality 1
4321
CMAF Fragments
Quality 2
CMAF
Header
CMAF
Header
21
CMAF
Header
43

19. Institut Mines-Télécom
CMAF Terminology
 Switching Sets
• Fragments from different Tracks that are switchable from
a user perspective (language, aspect, …)
• Switching Sets Types
─ Switching without pipeline re-initialization
• Single Initialization Switching Set
─ Switching with pipeline re-initialization
• Multiple Initialization Switching Set
─ Switching with relaxed constraints (e.g. encryption changes)
• Aligned Switching Sets
 CMAF Selection Sets
• Switching Sets where only one of them is displayed
 CMAF Presentation
• Selection Sets that form a presentation
Cyril Concolato - Paris Video Tech #319

20. Institut Mines-Télécom
CMAF Data Model
Cyril Concolato - Paris Video Tech #320
CMAF Presentation
CMAF Selection Set
CMAF Aligned Switching Sets
CMAF
Switching
Set
…
CMAF Switching Set
CMAF
Track
CMAF
Track
CMAF
Track
CMAF
Aligned
Switching
Sets
…
…
CMAF
Selection
Set
…
…
…
…
…
…
…

21. Institut Mines-Télécom
CMAF Profiles
Media Profiles
Presentation Profiles
Cyril Concolato - Paris Video Tech #321

22. Institut Mines-Télécom
AVC-based Video Media Profiles
Media
Profile
Cod
ec
Pro
file
Lev
el
Color
Coding
Transfer
Charact
eristics
Max
Fra
me
Heig
ht
Max
Fra
me
Widt
h
Max
Fra
me
Rat
e
CMAF
File
Brand
SD AVC Hig
h
3.1 BT.709
or
BT.601
BT.709
or
BT.601
OETF
576 864 60 'cfsd'
HD AVC Hig
h
4.0 BT.709 BT.709
OETF
1080 1920 60 'cfhd'
HDHF AVC Hig
h
4.2 BT.709 BT.709
OETF
1080 1920 60 'chdf'
Cyril Concolato - Paris Video Tech #322

23. Institut Mines-Télécom
HEVC-base Video Media Profiles
Media
Profile
Code
c
Profile Level Color
Coding
Transfer
Characterist
ics
Max
Frame
Height
Max
Frame
Width
Max
Frame
Rate
CMAF
File
Brand
HHD8 HEVC Main
MainTier
4.1 BT.709 BT.709
OETF
1080 1920 60 'chhd'
HHD10 HEVC Main10
MainTier
4.1 BT.709 BT.709
OETF
1080 1920 60 'chh1'
UHD8 HEVC Main8
MainTier
5.0 BT.709 BT.709
OETF
2160 3840 60 'cud8'
UHD10 HEVC Main10
MainTier
10-bit
5.1 BT.709,
BT.2020
BT.709
OETF,
BT.2020
OETF
2160 3840 60 'cud1'
HDR10 HEVC Main10
MainTier
10-bit
5.1 BT-2020 BT.2100
Table 4 PQ
EOTF
2160 3840 60 'cpq1'
HLG10 HEVC Main10
MainTier
10-bit
5.1 BT-2020 BT.2100
Table 5 HLG
OETF
[note1]
2160 3840 60 'clg1'
Cyril Concolato - Paris Video Tech #323

24. Institut Mines-Télécom
Audio Track Profiles
Media
Profile
Codec and Profile Number of
channels
Max
Sampling
Rate
File
Brand
AAC Core AAC-LC, HE-AAC or
HE-AAC v2
Mono or Stereo 48 kHz 'caac'
AAC
Adaptive
AAC Core
constrained for
adaptive switching
Mono or Stereo 48 kHz ‘caaa’
Cyril Concolato - Paris Video Tech #324

25. Institut Mines-Télécom
Subtitle Profiles
Media Profile Format Notes File
Brand
WebVTT WebVTT, Version 1.0 —pending normative
reference
https://w3c.github.io/w
ebvtt/
'cwvt'
TTML IMSC1
Text
Specified in Section
1.1.4
IMSC1 Text Profile 'im1t'
TTML IMSC1
Image
Specified in Section
1.1.5
IMSC1 Image Profile ‘im1i’
CEA CEA-608 and CEA-708 Caption data is
embedded in SEI
messages in video
track; multiple closed
caption streams may
be present1
'ccea'
Cyril Concolato - Paris Video Tech #325

26. Institut Mines-Télécom
CMAF Presentation Profiles
 urn:mpeg:cmaf:presentation_profile:cmfhd:2017
• At least ‘cfhd’
• At least ‘caac’
• At least ‘im1t’
• Not encrypted
 urn:mpeg:cmaf:presentation_profile:cmfhdc:2017
• CMFHD but with at least one ‘cenc’ encrypted media
 urn:mpeg:cmaf:presentation_profile:cmfhds:2017
• CMFHD but with at least one ‘cbcs’ encrypted media
Cyril Concolato - Paris Video Tech #326

27. Institut Mines-Télécom
Thank you. Questions ?
Cyril Concolato - Paris Video Tech #327