This presentation given by Zhaolin Wang, VIA Embedded Software Manager, provides an overview of the Android multimedia framework and then moves on to tackle critical areas such as enabling full support for RTSP and HTTP protocols, and optimizing the Android multimedia framework to match streaming server parameters, as well as shortening processing time (from start to initial playback) to ensure smooth playback, how to solve problems of clock drift (out of sync audio and video), and improving fault tolerance to avoid crashes and freezes due to corrupt data.

1. Embedded Android Real-Time
Streaming Optimization

2. Background
Android Multimedia
Framework Optimization
Real-Time Streaming &
Playback Optimization
Usage Scenarios

3. Courtroom Surveillance System
Courtroom

4. Background
Android Multimedia
Framework Optimization
Multimedia vs Streaming Media
AwesomePlayer Framework
NuPlayer Framework
Streaming Media Challenges

5. Multimedia vs Streaming Media
Multimedia is the integration of multiple forms of
media. This includes text, graphics, audio, video
Streaming media is video or audio content sent in
compressed form over the Internet and played
immediately, rather than being saved to the hard drive.

6. DemuxerSource
Video Decoder
Audio Decoder
Video Output
Audio Output
Decoder Output
Multimedia Decoding Process

7. Android Multimedia Framework

8. awesomeplayer
OMXCODEC
EMPTY_BUFFER_DONE
drainInputBuffer
FILL_BUFFER_DONE
fillOutputBuffer
OMX.Video.Decoder
read
OMX_EmptyThisBuffer OMX_FillThisBuffer
AwesomePlayer Framework

9. A
B
Message
1. post message1
2. call B.fun1() 3. post reply
4. call A.func2()
Acodec::onOMXEmptyBufferDone
OMX_EmptyThisBuffer
Message
1. post ACodec::kWhatFillThisBuffer
Nuplayer::feedDecoderInputData
2. call 3. post rely kWhatInputBufferFilled
Acodec::onInputBufferFilled
4. call
5. call
NuPlayer Framework

10.  Only supports RTSP and HTTP (No RTP/UDP support）
 Processing Time (Start to Initial Playback)
 Immediacy
 Instantaneity
 Clock drift
 Fault tolerance
 Poor Performance (multi-channel surveillance preview)
 Android Multimedia Player only supports preview/play
(snapshot, record, reply not supported)
Real-Time Streaming & Playback Challenges

11. Storage Introduction
Processing Time
Streaming Framework
Fault tolerance
Clock Drift
Immediacy
High Performance
Background
Android Multimedia Framework Optimization
Real-Time Streaming & Playback Optimization

12. RTSP Data Source
RTSP Media
Extrator
RTSP Storage
OMXCodec Output
Awesomeplayer
Streaming Framework

13. Record
Snapshot
SnapMovie (Real-time monitoring, playback)
Storage Introduction

14. Revised OMXCodec operating mode, quick creation
of media player clock
Improve RTSP connection procedure
Processing Time

15. High speed! It's simple to understand. It is how fast a
request can be processed!
Immediacy

16. Enhance errors at the receiving end of data,
decoder, display side of the fault-tolerant
processing
Fault Tolerance

17. NTP server
Streaming - Sound Synchronization
Clock Drift
Streaming - No Sound Synchronization
Cloud Server
PC LED Clock Router
Exchange
Data Server
GPS Antenna
GPS Satellite
Clock Server

18. High performance RTSP media
Completely disable Stagefright
4 frame 720p preview + 8 frame 1080p storage
High Performance I

19. 8 frame 360p preview + 8 frame 1080p storage
7 frame 270p preview + 1 frame 1080p preview
+ 8 frame 1080p storage
High Performance II

20. Background
Android Multimedia
Framework Optimization
Real-Time Streaming &
Playback Optimization
Usage Scenarios

21. Usage Scenarios

22. VIA
Strong
hardware
capabilities
Powerful
software
development
Customization
Ability to meet
customers
detailed
requests
Conclusion

23. © 2014 VIA Technologies, Inc All Rights Reserved.
• VIA reserves the right to make changes in its products without notice in order to improve design or performance characteristics.
• This publication neither states nor implies any representations or warranties of any kind, including but not limited to any implied warranty of merchantability or fitness for a particular
purpose. No license, express or implied, to any intellectual property rights is granted by this document.
• VIA makes no representations or warranties with respect to the accuracy or completeness of the contents of this publication or the information contained herein, and reserves the right
to make changes at any time, without notice. VIA disclaims responsibility for any consequences resulting from the use of the information included herein.
• VIA C7®, VIA C7®-D, VIA C7®-M, and VIA Eden® are trademarks of VIA Technologies, Inc.