The presentation illustrates how a Japanese streaming service handles chaotically supplied contents for 24/7 linear streaming. It also describes the background behind the fact that unique circumstances between the video streaming industry and broadcasting industry in Japan create the chaos.

1. A Japanese Way to Maintain Constant Quality
on Streaming Chaotically Supplied Contents
Yusuke Goto, AbemaTV, Inc.
- 2019.11.10 Video Dev Days 2019 -

2. Yusuke Goto
五藤 佑典
https://ygoto3.com/
@ygoto3_
● Majored in Graphic Design at California State University, San Bernardino
● Software engineer @ CyberAgent, Inc. and AbemaTV, Inc.
● AbemaTV Development Division
○ Lead New Device team
○ Lead Streaming Client team
○ A Developer Expert in video technology and product design

3. Everything about video tech in Japan is unique.
Unique practices, original specifications ...

4. AbemaTV
2.3 Tbps
(via Akamai’s Adaptive Media Delivery)
updated Japan’s highest traffic

5. 20 Channels * 24/7 + 18,000 Original Episodes
20 channels 24/7

6. We do
● Studio work
○ Shooting
○ Editing
● Production
○ Pre-production
○ Post-production
● Mastering
● Development
○ The streaming system
○ The applications
○ The ad system
● Ad production

7. Our workflow is complex ...

8. Recording Editing
Buying
MP4MXF
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4

9. Recording Editing
Buying
Converting
MP4MXF
MP4
MPEG 2
MOV
MXF
MP4
AVI
WMV
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4
Ad agents / Post-production
MP4
Ad
Operation
MP4

10. Recording Editing
Buying
Converting
MP4MXF
MP4
MPEG 2
MOV
MXF
MP4
AVI
WMV
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4
Ad agents / Post-production
MP4
Ad
Operation
MP4

11. If you don’t do anything ...

12. -24.0
LUFS
Linear Streaming and Server-Side Ad Insertion
CFR
Progressive
Interlaced
-18.0
LUFS
VFR
16:00 - 17:00
Slot A - Purchased
17:00 - 18:00
Slot B - Recorded
18:00 - 20:00
Slot C - Live
20:00 - 21:00
Slot D - Recorded

13. Example : AbemaTV’s Ad Regulation for Submission
Duration 15s/30s/60s/90s/120s/150s/180s/300s
Container Format MP4
Frame Size Full HD (1920 x 1080)
DAR 16:9
Video Bitrate (avg.) 20 Mbps
Video Encoding Mode 2-pass encoding
Scan Type Progressive scan
Video Codec H.264
Frame Rate 29.97 fps
Profile High Profile / Main Profile
Audio Channels Stereo 2ch
Audio Codec AAC-LC
Audio Bitrate 320 Kbps
Audio Bit depth 16 bits
Audio Sampling Freq 48 kHz
Audio Peak Level >= -3 dDBFS
Loudness (avg.) -18.0 LUFS ± 1LU

14. However, it’s not easy to follow the regulation

15. Sources Come from Everywhere
● Broadcasters
○ Their workflow is much simpler and
fixed
● Masters for BD/DVD
○ Too much info
● Old contents
○ Legacy codec
■ Can’t handle
w/ the latest software
● Sources originally for another purpose
○ Digital signage
○ Web video
■ Fewer framerate
■ VFR
○ Archives of game streaming video
○ Hardware-dependent codec
■ Via hardware-recording
● Multi-language sources
○ Multi-track
■ Primary tracks platform-dependent
● etc.
○ All the video sources that
are not optimized for streaming

16. Chaotic Sources
MP4
H.264
AAC-LC
MOV
ProRes 422
Linear PCM
MXF
XDCAM HD
422
Linear PCM
AVI
Canopus HQ
DV
Linear PCM
WMV
MPEG 2
MPEG 2
Linear PCM
MPEG Audio
Recording
Buying
Ad agents / Post-production

17. Mastering for AbemaTV
MPEG2
MXF
MOV
MP4
AVI
WMV
MPEG-2 TS
fMP4
Chaotic

18. Need a mezzanine

19. Mastering for AbemaTV
MPEG-2 TS
fMP4
MP4
(mezzanine)
MPEG2
MXF
MOV
MP4
AVI
WMV
Chaotic

20. Why MP4 ?
● Good for our needs
○ Small file size
○ Good compression for 4:2:0 chroma subsampling
■ Good enough as a de facto media format
○ Decodable everywhere
■ Playable everywhere
○ Fast transcoding
○ Simple structure
■ Easy to handle GOP
■ Easy to convert sources to I-frame-only data

21. Why MP4 ?
● MP4 already seems to be the standard for delivery
○ Major media solution providers primarily support MP4
■ AWS Elemental MediaLive
● mp4 only
■ Harmonic
● mov/mxf/mp4 - (mp4 is the stablest)
■ We do not use these solutions, but good to have other options

22. However, it’s not easy to generate a good-quality mp4

23. MP4 is hard to handle
● MP4 is an effectively compressed format
○ Easily succeeds negative factors from the original sources
■ Broken decode order
■ Wrong header info
Not all the content operators are video experts

24. Auto QC / Auto transcoding

25. Recording
Buying
MXF
MPEG 2
MOV
MXF
MP4
AVI
WMV
Ad agents / Post-production
MP4
Auto QC / Auto transcoding
through MAM

26. Recording
Buying
MXF
MPEG 2
MOV
MXF
MP4
AVI
WMV
Ad agents / Post-production
MP4
Auto QC / Auto transcoding
through MAM

27. A MAM solution is good, but ...
● Does not support all the media formats we receive
● MAM companies are usually not familiar with the Japanese manner
○ e.g., DALET is a French company
○ 30 fps / NTSC ? No, PAL does not use it

28. A semi-auto QC / semi-auto converting step before MAM

29. Recording
Buying
MXF
MPEG 2
MOV
MXF
MP4
AVI
WMV
Ad agents / Post-production
MP4
Auto QC / Auto transcoding
through MAM
Semi-auto QC / Semi-auto converting
Semi-auto QC / Semi-auto converting
Semi-auto QC / Semi-auto converting

31. Judging the Scan Type
Headers tell lies

32. Judging the Scan Type
Headers tell lies

33. Judging the Scan Type
ffmpeg -ss {duration/2} -t {duration*0.1} -vf idet ...
isInterlaced = (TFF + BFF) / (TFF + BFF) + Progressive > 0.2

34. -18.0
LUFS
Now it’s constant
CFR
Progressive
Progressive
-18.0
LUFS
CFR
16:00 - 17:00
Slot A - Purchased
17:00 - 18:00
Slot B - Recorded
18:00 - 20:00
Slot C - Live
20:00 - 21:00
Slot D - Recorded

35. Loudness
-18.0 LUFS
● AbemaTV’s original specification
○ ARIB T032 specifies -24.0 LUFS
■ Not enough for mobile devices
○ Optimized for playback on mobile devices
○ Follows AES’ Recommendation for Loudness of Audio Streaming &
Network File Playback
■ -16.0 to -20.0 LUFS

36. Various Kinds of Platforms to Support

37. Various Kinds of Platforms to Support
● iPhone
● iPad
● Apple TV
● Android Smartphone
● Android Tablet
● Android TV
● Fire Tablet
● Amazon Fire TV
● Google Chrome
● Mozilla Firefox
● Internet Explorer
● Microsoft Edge
● Apple Safari
● Google Chromecast
● Google Daydream View
● Amazon Echo
● LINE Clova Desk
● SONY BRAVIA
● Panasonic VIERA
● TOSHIBA REGZA
● Sharp AQUOS
● FUNAI
● LEOPALACE21
Life Stick
● CCCAIR Air Stick
● ...

38. Buffering Capacities Differ by Platform
Chrome Firefox Safari Edge Chromecast
Video 150 MB 100 MB 290 MB Unknown 30 MB
Audio 12 MB 15 MB 14 MB Unknown 2 MB
https://developers.google.com/web/updates/2017/10/quotaexceedederror

39. Buffering Capacities Differ by Platform
Chrome Firefox Safari Edge Chromecast
Video 150 MB 100 MB 290 MB Unknown 30 MB
Audio 12 MB 15 MB 14 MB Unknown 2 MB
https://developers.google.com/web/updates/2017/10/quotaexceedederror

40. QuotaExceededError
https://cs.chromium.org/chromium/src/media/base/demuxer_memory_limit.h

41. Beauty > Bitrate

42. Beauty > Bitrate
$ ffmpeg -maxrate ..k -bufsize ..k ..

45. CS workflow
Customer Support Player Dev.Users
Streaming Dev.
Content Ops

46. CS workflow
Customer Support Player Dev.Users
Streaming Dev.
Content Ops

47. Visualizing the Buffering Transition

48. df = pandas.DataFrame(keyframes)
...
ddf = df.diff()
df['diff'] = ddf['pos'] / ddf['time']
df.index = pandas.TimedeltaIndex([pandas.Timedelta(minutes=x) for x in df['time']])
up_sampled = df['pos'].resample('S').mean().interpolate('linear')
upx = up_sampled.copy(deep=True)
upx.index -= pandas.Timedelta(f'{buffer_length}s')
upx = upx[upx.index > pandas.Timedelta('0s')]
x = pandas.DataFrame()
x[f'{buffer_length} seconds'] = upx - up_sampled

49. CS workflow
Customer Support Debugging Player Dev.Users
Streaming Dev.
Content Ops

50. Conclusion
● To keep constant quality on streaming chaotically supplied contents
○ Developed a unique and original specifications
○ Partly adopt parts of various standardized specifications
○ Developed tools and workflow to patch gaps

51. Yusuke Goto
五藤 佑典
https://ygoto3.com/
@ygoto3_
Thank you !