Enhance Qualcomm Snapdragon Audio using Native Android Audio APIs
•
13 likes•14,441 views
This session will demonstrate how a developer can use standard android APIs to configure parameters of audio module/feature code running on Qualcomm® Hexagon™ DSP Core contained in Qualcomm® Snapdragon™ processors. The benefit is enhanced precision, sample rate, concurrency, and reduced power consumption while retaining standard Android APIs. Qualcomm Snapdragon and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. Watch this presentation on YouTube: https://www.youtube.com/watch?v=1PU0nFJHd1c
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (16)
More from Qualcomm Developer Network
More from Qualcomm Developer Network (20)
Recently uploaded
Recently uploaded (20)
Enhance Qualcomm Snapdragon Audio using Native Android Audio APIs
- 1. 1
- 2. Enhance Qualcomm® Snapdragon™ audio using Android Audio APIs Harikishan Desineni, Director of Engineering Qualcomm Innovation Center, Inc. Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc.
- 3. 3 Agenda Session overview 1 How Qualcomm Technologies incorporated the digital signal processor (DSP) 2 Snapdragon Audio SDK 3 Open DSP program 4 Demo 5
- 4. 4 Includes the following sound effects: − Equalizer (EQ) – Adjusts the sound level of certain frequency ranges by use of audio filters to improve the quality of overall perceived sounds − Virtualizer – Virtualizes audio channels to compensate for the mismatch between the loudspeaker setup used when producing the audio, and the audio output device used when the audio is played back by the end user − Reverb – Simulates the reverberation that results from reflections from surrounding walls and objects in an environment to make the sound more live and real − BassBoost – Dynamically boosts low-frequency content of an audio signal to achieve better listening experience on bass for a headset device How Qualcomm® Snapdragon™ Audio+ uses the DSP Qualcomm Snapdragon Audio+ is a product of Qualcomm Technologies, Inc.
- 5. 5 Media playback block diagram Surface Flinger Audio Flinger MDP/MDDI Media Player Media Player Service StageFright User space Java User space Native Kernel space DSP Video OMX Component Audio OMX Component Video HW PCM-audio Audio HAL Audio HW Media Server Audio DriversDisplay Drivers Video Drivers
- 6. 6 Audio framework: effects offloaded to DSP Media Player Media Player Service Awesome Player Audio Track AudioFlinger Qualcomm® Hexagon™ AudioEffects Software AudioEffect BassBosst Virtualizer ReverB X Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.
- 8. 8 Snapdragon Audio SDK Fluence™ Pro EC NS detection Snapdragon SDK for Android: Device Info The Device Info feature set is designed to allow an application to detect if the device it is running on has a Snapdragon processor. This insight lets you make informed decisions about how to take full advantage of features enabled by Snapdragon processors. Using Device Info is designed to let you tap into the following hardware features to help improve the performance of your applications: − Is Snapdragon This feature is designed to allow the application to detect at runtime if the device has a Snapdragon processor − Echo Cancellation/Noise Suppression Echo Cancellation/Noise Suppression is designed to foster better audio communication quality at lower power by allowing full duplex mode during VOIP calls. These features are specifically tuned to each device, and use technology available on normal mobile voice calls so you can offload CPU/software-based echo cancellation and noise suppression. Fluence Pro is a product of Qualcomm Technologies, Inc.
- 9. 9 Snapdragon Audio SDK com.qualcomm.snapdragon.sdk.deviceinfo Class Description DeviceInfo The DeviceInfo API provides a mechanism to query a device about the presence of certain functionalities. Class Description DeviceInfo.Features The list of features that can be detected with the function isFeatureSupported(Feature, Context) Sample code is subject to SDK license agreement. Source: http://developer.qualcomm.com
- 10. 10 Snapdragon Audio SDK public static final DeviceInfo.Features FEATURE_FLUENCE Noise Cancellation Technology − If isFeatureSupported returns true for this feature, then it can be assumed that echo cancellation is supported on the voice path. − If isFeatureSupported returns true for this feature, and the standard Android API android.media.audiofx.AcousticEchoCanceler.isAvailable() also returns true, then it can be assumed that echo cancellation is also available on the audio path.
- 11. 11 Snapdragon Audio SDK FEATURE_SURROUND_SOUND_RECORDING public static final DeviceInfo.Features FEATURE_SURROUND_SOUND_RECORDINGSurround Sound Recording is designed to allow for 5.1 surround sound input using existing microphones on the device
- 13. 13 Audio Modules – Common Interface Common interface for audio modules that are defined to ease integration with the Elite Framework Audio Postprocessor Interface (APPI) for audio processing algorithms Common Audio Processor Interface (CAPI) for audio codec modules Processing Algorithm Wrapper with common interface (APPI) Codec Decoder / Encoder Wrapper with common interface (CAPI) Note: The terms algorithm, module, and library are used interchangeably in this presentation.
- 14. 14 Audio Modules – Static and Dynamic Two types of modules, based on loading: − Static modules (*.lib) − Linked to the aDSP image at compile time − Symbol are resolved at compile time − aDSP image must be recompiled when a module changes − Dynamic modules (*.so) − Loaded at run-time into aDSP heap memory − Symbols are resolved when the module is loaded into memory − aDSP image does not need to be recompiled when a module changes Each module is identified by unique global module ID Audio static modules are added to the AMDB at boot time Audio dynamic modules are added to the AMDB at run time APPI mod-n (*.so) CAPI mod-3 (*.so) CAPI mod-2 (*.lib) APPI mod-1 (*.lib)
- 15. 15 Per Object PostProcessing (POPP) Per Object PreProcessing (POPreP ) Processing chain per audio stream Default POPP topologies: − Playback session − Record session DSP audio mixer Downmix if needed Resampler Audio EncoderVolume DSP audio mixer Volume Audio DecoderResampler POPP Topologies
- 16. 16 Common Object PostProcessing (COPP) Common Object PreProcessing (COPreP) Processing chain per device Default COPP topologies: − Playback session − Record session AFE Audio pregain AFE CodecRX Gain DSP Audio Mixer Audio Volume DSP Audio Mixer COPP Topologies
- 17. 17 Custom Audio Topology – Static and Dynamic Two types of audio topologies: − Static topologies − Created at compile time − aDSP image must be recompiled if the topology changes − Dynamic topology − Created at run time − aDSP image does not need to be recompiled if the topology changes − APIs are defined for adding custom dynamic topologies from the applications processor: Custom topology is designed to use any module defined in the AMDB by using Module IDs Customized topology with an OEM algorithm ASM_CMD_ADD_TOPOLOGIES // for POPP and POPREP ADM_CMD_ADD_TOPOLOGIES_V5 // for COPP and COPREP Audio Mixer Volume OEM Algorithm Audio DecoderResampler
- 18. 18 Audio Dynamic Topology How are dynamic topologies added at run time? APR ADM Topo DB ASM_CMD_ADD_TOPOLOGIES T2 (-M1-M4-), T3 (-M1-M2) topo_db_add_topologies() T1 T2 T3 aDSP AMDB M1 M2 M3 M4 New audio sessions can be created using T2 and T3 topologies after adding them to the topology databaseClient Processor ASM
- 19. 19 ISV Module1 ISV Module2 Application HAL RTAC Kernel Driver APR Handler DSP Payload ASM_CMD_SET_PP_PARAMS/ AMD_CMD_SET_PP_PARAMS RTAC Userspace module Send_rtac() ACDB Loader Acdb_store() Acdb_send() Acdb_get() acdb file Get/setParameter(key”,value) Software Architecture – Block Diagram
- 20. 20 Software Architecture – Android APIs Existing set parameter android API extended to have new key value pairs Setparameter() with persistence − set_parameter(key=”ISV_PERSIST_CFG”, value=” DataBinary, *”); − Data Binary – Entire calibration data binary that is stored in calibration database. Even if user changes single parameter through UI, the app is responsible for packaging the entire data binary for persistence Setparameter() without persistence − set_parameter(key=”ISV_NOPERSIST_CFG”, value, “List_MID_PID, *”); − List_MID_PID – List of dynamically modified calibration to be applied on ISV modules in ADSP. This could be an array of Module ID, Parameter ID and data, or could be only one set of ModuleId, Parameter ID and values. Here, the data is not persisted in audio calibration database. The app is responsible for packaging the modified modules and parameters.
- 21. 21 Demo
- 22. 22 developer.qualcomm.com Augmented reality Context awareness Peer to peer Wireless health Graphics & gaming Computer vision Android applications App development SDKs & tools Marketing opportunities Case Studies & Tutorials News & Information Hardware development platforms Forums & technical support Tools & resources to help developers build, integrate, and optimize Qualcomm Developer Network
- 23. 23 For more information on Qualcomm, visit us at: www.qualcomm.com & www.qualcomm.com/blog Thank you FOLLOW US ON: © 2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All rights reserved. Qualcomm, Snapdragon, Hexagon and Fluence are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Uplinq is a trademark of Qualcomm Incorporated. All trademarks of Qualcomm Incorporated are used with permission. Other product and brand names may be trademarks or registered trademarks of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.