Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Embedded Android Real-Time Streaming Optimization

This presentation given by Zhaolin Wang, VIA Embedded Software Manager, provides an overview of the Android multimedia framework and then moves on to tackle critical areas such as enabling full support for RTSP and HTTP protocols, and optimizing the Android multimedia framework to match streaming server parameters, as well as shortening processing time (from start to initial playback) to ensure smooth playback, how to solve problems of clock drift (out of sync audio and video), and improving fault tolerance to avoid crashes and freezes due to corrupt data.

  • Login to see the comments

Embedded Android Real-Time Streaming Optimization

  1. 1. Embedded Android Real-Time Streaming Optimization
  2. 2. Background Android Multimedia Framework Optimization Real-Time Streaming & Playback Optimization Usage Scenarios
  3. 3. Courtroom Surveillance System Courtroom
  4. 4. Background Android Multimedia Framework Optimization Multimedia vs Streaming Media AwesomePlayer Framework NuPlayer Framework Streaming Media Challenges
  5. 5. Multimedia vs Streaming Media Multimedia is the integration of multiple forms of media. This includes text, graphics, audio, video Streaming media is video or audio content sent in compressed form over the Internet and played immediately, rather than being saved to the hard drive.
  6. 6. DemuxerSource Video Decoder Audio Decoder Video Output Audio Output Decoder Output Multimedia Decoding Process
  7. 7. Android Multimedia Framework
  8. 8. awesomeplayer OMXCODEC EMPTY_BUFFER_DONE drainInputBuffer FILL_BUFFER_DONE fillOutputBuffer OMX.Video.Decoder read OMX_EmptyThisBuffer OMX_FillThisBuffer AwesomePlayer Framework
  9. 9. A B Message 1. post message1 2. call B.fun1() 3. post reply 4. call A.func2() Acodec::onOMXEmptyBufferDone OMX_EmptyThisBuffer Message 1. post ACodec::kWhatFillThisBuffer Nuplayer::feedDecoderInputData 2. call 3. post rely kWhatInputBufferFilled Acodec::onInputBufferFilled 4. call 5. call NuPlayer Framework
  10. 10.  Only supports RTSP and HTTP (No RTP/UDP support)  Processing Time (Start to Initial Playback)  Immediacy  Instantaneity  Clock drift  Fault tolerance  Poor Performance (multi-channel surveillance preview)  Android Multimedia Player only supports preview/play (snapshot, record, reply not supported) Real-Time Streaming & Playback Challenges
  11. 11. Storage Introduction Processing Time Streaming Framework Fault tolerance Clock Drift Immediacy High Performance Background Android Multimedia Framework Optimization Real-Time Streaming & Playback Optimization
  12. 12. RTSP Data Source RTSP Media Extrator RTSP Storage OMXCodec Output Awesomeplayer Streaming Framework
  13. 13. Record Snapshot SnapMovie (Real-time monitoring, playback) Storage Introduction
  14. 14. Revised OMXCodec operating mode, quick creation of media player clock Improve RTSP connection procedure Processing Time
  15. 15. High speed! It's simple to understand. It is how fast a request can be processed! Immediacy
  16. 16. Enhance errors at the receiving end of data, decoder, display side of the fault-tolerant processing Fault Tolerance
  17. 17. NTP server Streaming - Sound Synchronization Clock Drift Streaming - No Sound Synchronization Cloud Server PC LED Clock Router Exchange Data Server GPS Antenna GPS Satellite Clock Server
  18. 18. High performance RTSP media Completely disable Stagefright 4 frame 720p preview + 8 frame 1080p storage High Performance I
  19. 19. 8 frame 360p preview + 8 frame 1080p storage 7 frame 270p preview + 1 frame 1080p preview + 8 frame 1080p storage High Performance II
  20. 20. Background Android Multimedia Framework Optimization Real-Time Streaming & Playback Optimization Usage Scenarios
  21. 21. Usage Scenarios
  22. 22. VIA Strong hardware capabilities Powerful software development Customization Ability to meet customers detailed requests Conclusion
  23. 23. © 2014 VIA Technologies, Inc All Rights Reserved. • VIA reserves the right to make changes in its products without notice in order to improve design or performance characteristics. • This publication neither states nor implies any representations or warranties of any kind, including but not limited to any implied warranty of merchantability or fitness for a particular purpose. No license, express or implied, to any intellectual property rights is granted by this document. • VIA makes no representations or warranties with respect to the accuracy or completeness of the contents of this publication or the information contained herein, and reserves the right to make changes at any time, without notice. VIA disclaims responsibility for any consequences resulting from the use of the information included herein. • VIA C7®, VIA C7®-D, VIA C7®-M, and VIA Eden® are trademarks of VIA Technologies, Inc.

×