Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Team Emertxe
Android System Development
Day-3
● Android Audio HAL
– Audio Architecture
– Audio HAL interface
– Audio Policy
– Audio HAL compilation & verification
– Ove...
Audio HAL
Audio HAL
(Overview)
● Connects higher-level, audio-specific framework APIs in
android.media to the underlying audio drive...
Audio Architecture
*source : Android documentation
Audio Architecture
Application
Application framework
Native Framework
Audio HAL
Tiny Alsa
Java
C++
C
Kernel ALSA
Audio Architecture
(Native Framework)
Application framework
Native Framework
Audio HAL
Tiny Alsa
AudioTrack AudioRecord Au...
Audio HAL
(Implementation steps)
● Implement interfaces described in audio.h,
audio_effect.h files
● Create an audio polic...
Audio HAL
(Interfaces)
● Main functions -
– hardware/libhardware/include/hardware/audio.h
– audio_stream_t
– stream_callba...
Audio HAL
(Interfaces)
● Reference implementation -
– system/media/audio/include/system/audio.h
– device/samsung/tuna/audio
Audio HAL
(Audio policy configuration)
● Naugat introduced new XML based audio policy
– New audio policy configuration fil...
Audio HAL
(Audio policy – non-XML sample)
global_configuration {
attached_output_devices AUDIO_DEVICE_OUT_SPEAKER
default_...
Audio HAL
(Audio policy configuration)
● XML files from other folders can be included using Xinclude element
– Example : <...
Audio HAL
(Audio policy – XML sample)
<audioPolicyConfiguration version="1.0">
<globalConfiguration speaker_drc_enabled="t...
Audio HAL
(Advantages of XML based policy)
● Audio profiles are now structured similar to HDMI Simple Audio Descriptors
an...
Audio HAL
(shared library)
● Create a device/<company>/<device>/audio directory to
contain your library's source files.
● ...
Audio HAL
(Preprocessing Effects)
● Default pre-processing effects applied for each AudioSource are specified
in the /syst...
Audio HAL
(Module)
struct audio_module HAL_MODULE_INFO_SYM = {
.common = {
.tag = HARDWARE_MODULE_TAG,
.module_api_version...
Audio HAL
(hw_module_methods_t)
static struct hw_module_methods_t hal_module_methods = {
.open = adev_open,
};
TinyALSA
Audio HAL
(tinyalsa)
● ALSA is an open source sound architecture for Linux
● TinyALSA is a small ALSA library aimed to be ...
TinyALSA vs ALSA
# Feature TinyALSA ALSA
1 Control Interface Partly support Full Support
2 Mixer Interface Partly support ...
Audio HAL
(tinyalsa PCM APIs)
*Path: external/tinyalsa/include/tinyalsa/asoundlib.h
# API Description
1 pcm_open () Open P...
Audio HAL
(tinyalsa Parameter APIs)
*Path: external/tinyalsa/include/tinyalsa/asoundlib.h
# API Description
1 pcm_params_g...
Audio HAL
(tinyalsa MMAP APIs)
*Path: external/tinyalsa/include/tinyalsa/asoundlib.h
# API Description
1 pcm_mmap_write ()...
Audio HAL
(tinyalsa Other APIs)
*Path: external/tinyalsa/include/tinyalsa/asoundlib.h
# API Description
1 pcm_get_error ()...
Audio HAL
(Hands-on : configure audio)
● $ tinymix [Press Enter Key]
Mixer name: 'bcm2835 ALSA'
Number of controls: 6
ctl ...
Audio HAL
(Hands-on : configure audio)
● Set volume
– $ tinymix 0 50
● Set route
– $ tinymix 2 1
● $ adb push <media-file-...
Camera HAL
Camera HAL
(Overview)
● Connects the higher level camera framework APIs in
android.hardware to underlying camera driver an...
Camera Architecture
Camera API (v1/v2)
Native API (JNI) / Native Services
Native Camera Library
Camera Services
Camera HAL...
Camera Architecture
(APIs)
Camera API (v1/v2)
Android.hardware
.camera
Android.media
.MediaRecorder
Android.hardware
.Came...
Camera Architecture
(Native Camera Library)
JNI Camera Context
Native Camera Library
ICameraClient
Camera
ICamera
CameraBa...
Camera Architecture
(Camera Service)
Camera
Camera Services
Camera HAL
Device v1
ICameraClient ICameraDevice
callbacks
ICa...
Camera Architecture
(Camera Services)
Camera Hardware
Interface
Camera HAL
Camera HAL
Device v1
Camera HAL
Device v2
Camer...
Camera HAL
(Implementation)
● HAL sits between camera driver and Android framework
● HAL interface is defined in :
– hardw...
Camera HAL
(Creating shared library)
● Create a device/<company_name>/<device_name>/camera
directory to contain library's ...
Camera HAL
(Creating shared library)
● Declare camera’s media codec, format, and resolution capabilities in
– device/<comp...
Camera HAL
(v1, V2, V3)
● Supported camera HAL1 as many devices still rely on it
● Android camera service supports impleme...
Camera3
● Re-designed to
– Increase the ability of app to control camera
– Be more efficient and maintainable
● Additional...
Camera3
● Single unified view results in better user
control for
– Focus and exposure
– Post-processing, such as noise red...
Camera3
HAL operation
● Asynchronous requests for captures come from the framework
● HAL device must process requests in order
● F...
Camera3 operation
Framework operation
● Framework opens device
– camera_module_t common.open()→
● Framework checks version field & instantia...
Framework operation
● Framework requests default settings for some number of use cases with
calls to camera3_device_t ops ...
Camera operation flow
Camera Devices
● LEGACY : These devices expose capabilities to apps through the
Camera API2 interfaces that are approximat...
Camera modes
● The camera 3 HAL device can implement one of two possible
operational modes
– Limited
– Full
● Full support...
Camera HAL
(versions)
● Camera API1
– The app-level camera framework on Android 4.4 and earlier devices, exposed through
t...
Camera HAL
(Hands-on : enabling v4l2)
● $ make ARCH=arm menuconfig
● Select “Device Drivers”
● Select “Multimedia Support”...
Camera HAL
(Hands-on : enabling v4l2)
● $ ARCH=arm CROSS_COMPILE=arm-linux-androideabi- make zImage
-j4
● $ ARCH=arm CROSS...
Enabling Camera
● Add following parameters in /boot/config.txt
– start_x=1
– gpu_mem=256 (128 or more)
● Ensure start_x.el...
Camera Interface
(camera_device_ops_t)
● set_preview_window
● set_callbacks (notification and data)
● enable_msg_type
● di...
Camera Interface
(camera_device_ops_t)
● start_recording
● stop_recording
● recording_enabled
● release_recording_frame
● ...
Camera Interface
(camera_device_ops_t)
● set_parameters
● get_parameters
● put_parameters (to return memory to HAL)
● send...
Camera Interface
(preview_stream_ops_t)
● deque_buffer
● enqueue_buffer
● cancel_buffer
● set_buffer_count
● set_buffers_g...
Video4Linux (V4L2)
What is V4L2?
● An open source standard to capture real time
video for linux systems
● Second version of video for linux
●...
V4L2
(Devices)
Device Name Minor Number Description
From To
/dev/video0 /dev/video63 0-63 Video Capturer Devices
/dev/radi...
V4L2
(Video Device)
# Members Description
char name[32] Name of the device
int type Type of V4L2 device
int minor Device m...
V4L2
(File operations)
# Funcrion Description
1 int (*open)() Called when a file descriptor is opened on the
device file (...
V4L2
(Property Negotiation)
● First the application asks for the possibles values
of some property
● Next chooses one of t...
V4L2
(Pixel format negotiation)
● Pixel format is how every pixel is stored in memory
● Application need to know this form...
V4l2
(Buffers)
● V4L2_MMAP
– Memory mapping (allocated by the kernel)
● V4L2_USERPTR
– User memory (allocated by user app)...
V4L2
(Driver Compilation)
● Step 1 : Copy driver source files in following folder
– kernels/rpi/deriver/media/platform/<de...
V4L2
(Driver Compilation and loading)
● Step 4 : make ARCH=arm menuconfig
– Select driver with m option; save and exit
● S...
Stay connected
About us: Emertxe is India’s one of the top IT finishing schools & self learning
kits provider. Our primary...
Thank You
Upcoming SlideShare
Loading in …5
×

of

Embedded Android : System Development - Part III (Audio / Video HAL) Slide 1 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 2 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 3 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 4 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 5 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 6 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 7 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 8 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 9 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 10 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 11 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 12 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 13 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 14 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 15 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 16 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 17 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 18 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 19 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 20 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 21 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 22 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 23 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 24 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 25 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 26 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 27 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 28 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 29 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 30 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 31 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 32 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 33 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 34 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 35 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 36 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 37 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 38 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 39 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 40 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 41 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 42 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 43 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 44 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 45 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 46 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 47 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 48 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 49 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 50 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 51 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 52 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 53 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 54 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 55 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 56 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 57 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 58 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 59 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 60 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 61 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 62 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 63 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 64 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 65 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 66 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 67 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 68 Embedded Android : System Development - Part III (Audio / Video HAL) Slide 69
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

20 Likes

Share

Download to read offline

Embedded Android : System Development - Part III (Audio / Video HAL)

Download to read offline

Android Audio HAL – Audio Architecture – Audio HAL interface – Audio Policy – Audio HAL compilation & verification – Overview of Tinyalsa
Android Video HAL – Camera Architecture – Overview of camera HAL interface – Overview of V4L2 – Enabling V4l2 in kernel – Camera HAL compilation and verification

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Embedded Android : System Development - Part III (Audio / Video HAL)

  1. 1. Team Emertxe Android System Development Day-3
  2. 2. ● Android Audio HAL – Audio Architecture – Audio HAL interface – Audio Policy – Audio HAL compilation & verification – Overview of Tinyalsa ● Android Video HAL – Camera Architecture – Overview of camera HAL interface – Overview of V4L2 – Enabling V4l2 in kernel – Camera HAL compilation and verification Table of Content
  3. 3. Audio HAL
  4. 4. Audio HAL (Overview) ● Connects higher-level, audio-specific framework APIs in android.media to the underlying audio driver and hardware ● Defines standard interface that audio services call into and that you must implement for your audio hardware to function correctly ● Interfaces are located in hardware/libhardware/include/hardware
  5. 5. Audio Architecture *source : Android documentation
  6. 6. Audio Architecture Application Application framework Native Framework Audio HAL Tiny Alsa Java C++ C Kernel ALSA
  7. 7. Audio Architecture (Native Framework) Application framework Native Framework Audio HAL Tiny Alsa AudioTrack AudioRecord AudioEffect Audio Flinger AudioPolicy Service
  8. 8. Audio HAL (Implementation steps) ● Implement interfaces described in audio.h, audio_effect.h files ● Create an audio policy configuration file – Describe audio topology and package the HAL implementation into a shared library ● Configure pre-processing effects such as automatic gain control and noise suppression
  9. 9. Audio HAL (Interfaces) ● Main functions - – hardware/libhardware/include/hardware/audio.h – audio_stream_t – stream_callback_t – audio_stream_out_t – audio_stream_in_t – audio_hw_device_t ● Effects (downmixing, echo, or noise suppression) – hardware/libhardware/include/hardware/audio_effect.h – audio_effect_library_t – struct effect_interface_s
  10. 10. Audio HAL (Interfaces) ● Reference implementation - – system/media/audio/include/system/audio.h – device/samsung/tuna/audio
  11. 11. Audio HAL (Audio policy configuration) ● Naugat introduced new XML based audio policy – New audio policy configuration file audio_policy_configuration.xml – Location : /system/etc – Include build option USE_XML_AUDIO_POLICY_CONF := 1 in device makefile ● Old policy (audio_policy.conf) is deprecated – Location : device/<company>/<device>/audio/audio_policy.conf – Example : device/samsung/tuna/audio/audio_policy.conf
  12. 12. Audio HAL (Audio policy – non-XML sample) global_configuration { attached_output_devices AUDIO_DEVICE_OUT_SPEAKER default_output_device AUDIO_DEVICE_OUT_SPEAKER } audio_hw_modules { primary { outputs { primary { sampling_rates 44100 channel_masks AUDIO_CHANNEL_OUT_STEREO formats AUDIO_FORMAT_PCM_16_BIT devices AUDIO_DEVICE_OUT_SPEAKER flags AUDIO_OUTPUT_FLAG_PRIMARY } } } } *frameworks/av/services/audiopolicy/audio_policy.conf
  13. 13. Audio HAL (Audio policy configuration) ● XML files from other folders can be included using Xinclude element – Example : <xi:include href=”a2dp_audio_policy_configuration.xml”/> ● Audio policy files – A2DP: a2dp_audio_policy_configuration.xml – Reroute submix: rsubmix_audio_policy_configuration.xml – USB: usb_audio_policy_configuration.xml
  14. 14. Audio HAL (Audio policy – XML sample) <audioPolicyConfiguration version="1.0"> <globalConfiguration speaker_drc_enabled="true"/> <module name="primary" halVersion="3.0"> <attachedDevices> <item>Speaker</item> <item>Built-In Mic</item> </attachedDevices> <defaultOutputDevice>Speaker</defaultOutputDevice> </module> <xi:include href="audio_policy_volumes.xml"/> <xi:include href="default_volume_tables.xml"/> </audioPolicyConfiguration> *frameworks/av/services/audiopolicy/config/audio_policy_configuration.xml
  15. 15. Audio HAL (Advantages of XML based policy) ● Audio profiles are now structured similar to HDMI Simple Audio Descriptors and enable a different set of sampling rates/channel masks for each audio format ● Explicit definitions of all possible connections between devices and streams – Previously, an implicit rule made it possible to interconnect all devices attached to the same HAL module, preventing audio policy from controlling connections requested with audio patch APIs – In XML format, topology description now defines connection limitations ● Support for includes avoids repeating standard A2DP, USB, or reroute submit definitions ● Customizable volume curves – Previously, volume tables were hard-coded – In XML format, volume tables are described and can be customized
  16. 16. Audio HAL (shared library) ● Create a device/<company>/<device>/audio directory to contain your library's source files. ● Create an Android.mk file to build the shared library ● Make file shall contain – LOCAL_MODULE := audio.primary.<device> ● Create device.mk (see rpi3.mk in device/brcm/rpi/ directory)
  17. 17. Audio HAL (Preprocessing Effects) ● Default pre-processing effects applied for each AudioSource are specified in the /system/etc/audio_effects.conf file ● To apply custom effects for every AudioSource, create a /system/vendor/etc/audio_effects.conf file and specify the pre-processing effects to turn on
  18. 18. Audio HAL (Module) struct audio_module HAL_MODULE_INFO_SYM = { .common = { .tag = HARDWARE_MODULE_TAG, .module_api_version = AUDIO_MODULE_API_VERSION_0_1, .hal_api_version = HARDWARE_HAL_API_VERSION, .id = AUDIO_HARDWARE_MODULE_ID, .name = "Raspberry Pi Audio HW HAL", .author = "The Android Open Source Project", .methods = &hal_module_methods, }, }; ● Audio module shall be initialized as mentioned below ● Name shall be HAL_MODULE_INFO_SYM ● The hal_module_methods points to open function
  19. 19. Audio HAL (hw_module_methods_t) static struct hw_module_methods_t hal_module_methods = { .open = adev_open, };
  20. 20. TinyALSA
  21. 21. Audio HAL (tinyalsa) ● ALSA is an open source sound architecture for Linux ● TinyALSA is a small ALSA library aimed to be used in Android ● Source code is available in “external/tinyalsa” directory ● Utility executables – tinyplay – tinymix – tinycap – Tinypcminfo ● Shared library - libtinyalsa
  22. 22. TinyALSA vs ALSA # Feature TinyALSA ALSA 1 Control Interface Partly support Full Support 2 Mixer Interface Partly support Full Support 3 PCM Interface Partly support Full Support 4 Raw MIDI Interface No Full Support 5 Sequencer Interface No Full Support 6 Timer Interface No Full Support
  23. 23. Audio HAL (tinyalsa PCM APIs) *Path: external/tinyalsa/include/tinyalsa/asoundlib.h # API Description 1 pcm_open () Open PCM device 2 pcm_read () Read PCM audio data from PCM 3 pcm_write () Write data. This is start playback at first write. 4 pcm_close () Close driver 5 pcm_set_config () Set configuration parameters 6 pcm_get_config () Get configuration parameters 7 pcm_is_ready () Check if PCM driver is ready
  24. 24. Audio HAL (tinyalsa Parameter APIs) *Path: external/tinyalsa/include/tinyalsa/asoundlib.h # API Description 1 pcm_params_get () Get parameters 2 pcm_params_get_mask () Get mask 3 pcm_params_get_min () Get min 4 pcm_params_set_min () Set Min 5 pcm_params_get_max () Get Max 6 pcm_params_set_max () Set Max 7 pcm_params_free () Free memory allocated for parameters 8 pcm_params_to_string () Human readable parameter string 9 pcm_params_format_test ()
  25. 25. Audio HAL (tinyalsa MMAP APIs) *Path: external/tinyalsa/include/tinyalsa/asoundlib.h # API Description 1 pcm_mmap_write () Write to shared memory 2 pcm_mmap_read () Read from shared memory 3 pcm_mmap_begin () Request to access a portion of direct (mmap) area 4 pcm_mmap_commit () Completed the access to area requested (using begin) 5 pcm_mmap_avail () Number of frames ready to be read (capture) / written (playback) 8 pcm_prepare () Prepare the PCM substream to be triggerable 9 pcm_start () Start a PCM channel that doesn't transfer data 10 pcm_stop () Stop a PCM channel that doesn't transfer data 11 pcm_ioctl () ioctl function for PCM driver 12 pcm_wait () Interrupt driven API 13 pcm_get_poll_fd () Get poll file descriptor
  26. 26. Audio HAL (tinyalsa Other APIs) *Path: external/tinyalsa/include/tinyalsa/asoundlib.h # API Description 1 pcm_get_error () Returns human readable reason (string) for the last error 2 pcm_format_to_bits () Returns the sample size in bits for a PCM format 3 pcm_get_buffer_size () buffer size that should be used for pcm_write 4 pcm_frames_to_bytes () buffer size that should be used for pcm_write 5 pcm_bytes_to_frames () buffer size that should be used for pcm_write 6 pcm_get_latency () Get the PCM latency in milliseconds 7 pcm_get_htimestamp () Returns available frames in pcm buffer to read (in stream) or write (out stream)and corresponding time stamp
  27. 27. Audio HAL (Hands-on : configure audio) ● $ tinymix [Press Enter Key] Mixer name: 'bcm2835 ALSA' Number of controls: 6 ctl type num name value 0 INT 1 PCM Playback Volume 0 1 BOOL 1 PCM Playback Switch On 2 INT 1 PCM Playback Route 0 3 IEC958 1 IEC958 Playback Default unknown 4 IEC958 1 IEC958 Playback Con Mask unknown 5 IEC958 1 IEC958 Playback PCM Stream unknown
  28. 28. Audio HAL (Hands-on : configure audio) ● Set volume – $ tinymix 0 50 ● Set route – $ tinymix 2 1 ● $ adb push <media-file-name> /data/ ● Play media file – $ tinyplay <media-file-name>.wav
  29. 29. Camera HAL
  30. 30. Camera HAL (Overview) ● Connects the higher level camera framework APIs in android.hardware to underlying camera driver and hardware ● The camera HAL provides interfaces for use in implementing camera pipeline components ● Refer to following files – camera.h source file – camera3.h source file – camera_common.h source file
  31. 31. Camera Architecture Camera API (v1/v2) Native API (JNI) / Native Services Native Camera Library Camera Services Camera HAL Media Server Process Java C++ C
  32. 32. Camera Architecture (APIs) Camera API (v1/v2) Android.hardware .camera Android.media .MediaRecorder Android.hardware .Camera2 .CameraDevice Android.hardware .Camera2 .CameraManager Android.hardware .Camera2 .ICameraDeviceUser Android.hardware .Camera2 .ICameraService ICameraService CameraService Native Camera Library Camera Service CameraDeviceClient JNI Camera Context Stagefright Native API Native Services
  33. 33. Camera Architecture (Native Camera Library) JNI Camera Context Native Camera Library ICameraClient Camera ICamera CameraBase ICameraService Stagefright CameraService android.hardware .Camera2 .CameraManager android.hardware .ICameraService
  34. 34. Camera Architecture (Camera Service) Camera Camera Services Camera HAL Device v1 ICameraClient ICameraDevice callbacks ICameraService Listner CameraDevice Client Camera2ClientCameraClient CameraService CameraHardware Interface Camera2Device Camera3Device Camera HAL Device v2 Camera HAL Device v3 Camera HAL Module Camera APIsICamera
  35. 35. Camera Architecture (Camera Services) Camera Hardware Interface Camera HAL Camera HAL Device v1 Camera HAL Device v2 Camera HAL Device v3 Camera HAL Module Camera2Device Camera3Device Camera Service V4L2 (Linux Kernel)
  36. 36. Camera HAL (Implementation) ● HAL sits between camera driver and Android framework ● HAL interface is defined in : – hardware/libhardware/include/hardware/camera.h – hardware/libhardware/include/hardware/camera_common.h ● Camera_common.h defines camera_module, a standard structure to obtain general information about the camera ● Declares a camera_device struct that in turn contains a camera_device_ops struct with pointers to functions that implement the HAL interface ● Camera parameters are defined in frameworks/av/include/camera/CameraParameters.h
  37. 37. Camera HAL (Creating shared library) ● Create a device/<company_name>/<device_name>/camera directory to contain library's source files ● Create an Android.mk file to build the shared library ● Make file shall contain – LOCAL_MODULE := camera.<device_name> – LOCAL_MODULE_RELATIVE_PATH := hw ● Specify camera features by copying the necessary feature XML files in frameworks/native/data/etc directory with device.mk ● See reference file “device/samsung/tuna/device.mk”
  38. 38. Camera HAL (Creating shared library) ● Declare camera’s media codec, format, and resolution capabilities in – device/<company_name>/<device_name>/media_profiles.xml – device/<company_name>/<device_name>/media_codecs.xml XML files ● Add media_profiles.xml and media_codecs.xml in device.mk ● Add Camera app in PRODUCT_PACKAGES variable in device.mk to be part of system image
  39. 39. Camera HAL (v1, V2, V3) ● Supported camera HAL1 as many devices still rely on it ● Android camera service supports implementing both HALs (1 & 3) ● Useful when to support a less-capable front-facing camera with camera HAL1 ● Camera HAL2 is not supported as it was a temporary step on the way to camera HAL3 ● Single camera HAL module which lists multiple independent camera devices (each may have own version number) ● Camera module 2 or newer required to support devices 2 or newer
  40. 40. Camera3 ● Re-designed to – Increase the ability of app to control camera – Be more efficient and maintainable ● Additional camera control enables – Develop high-quality camera app – Operate reliably across multiple products – Use device-specific algorithms whenever possible & maximize quality & performance ● Structures the operation modes into a single unified view
  41. 41. Camera3 ● Single unified view results in better user control for – Focus and exposure – Post-processing, such as noise reduction, contrast and sharpening ● This simplified view makes it easier for app developers to use the camera's various functions
  42. 42. Camera3
  43. 43. HAL operation ● Asynchronous requests for captures come from the framework ● HAL device must process requests in order ● For each request, produce output result metadata, & one or more output image buffers ● First-in, first-out for requests & results, & for streams referenced by subsequent requests ● Timestamps must be identical for all outputs from a given request, so that the framework can match them together if needed ● All capture configuration and state [except for 3A (auto-exposure, auto-white-balance, auto-focus) control routines] is encapsulated in the requests and results
  44. 44. Camera3 operation
  45. 45. Framework operation ● Framework opens device – camera_module_t common.open()→ ● Framework checks version field & instantiates appropriate handler for that version of camera hardware device – camera3_device_t ops initialize()→ → ● Framework configure streams – camera3_device_t ops configure_streams()→ → ● Framework allocate stream buffers – camera3_device_t ops register_stream_buffers()→ →
  46. 46. Framework operation ● Framework requests default settings for some number of use cases with calls to camera3_device_t ops construct_default_request_settings()→ → ● Framework construct a capture request and send it to HAL – camera3_device_t ops process_capture_request()→ → ● HAL notifies framework of a started capture request – camera3_callback_ops notify()→ ● HAL notifies framework of a finished capture request – camera3_callback_ops process_capture_result()→ ● Framework calls close the device – camera3_device_t common close()→ →
  47. 47. Camera operation flow
  48. 48. Camera Devices ● LEGACY : These devices expose capabilities to apps through the Camera API2 interfaces that are approximately the same capabilities as those exposed to apps through the Camera API1 interfaces. The legacy frameworks code conceptually translates Camera API2 calls into Camera API1 calls; legacy devices do not support Camera API2 features such as per-frame controls ● FULL : These devices support all of major capabilities of Camera API2 and must use Camera HAL 3.2 or later and Android 5.0 or later ● LIMITED : These devices support some Camera API2 capabilities (but not all) and must use Camera HAL 3.2 or later
  49. 49. Camera modes ● The camera 3 HAL device can implement one of two possible operational modes – Limited – Full ● Full support is expected from new higher-end devices ● Limited mode has hardware requirements roughly in line with those for a camera HAL device v1 implementation, and is expected from older or inexpensive devices ● HAL must indicate its level of support with the android.info.supportedHardwareLevel static metadata entry, with 0 indicating limited mode, and 1 indicating full mode support.
  50. 50. Camera HAL (versions) ● Camera API1 – The app-level camera framework on Android 4.4 and earlier devices, exposed through the android.hardware.Camera class ● Camera API2 – The app-level camera framework on Android 5.0 and later devices, exposed through the android.hardware.camera2 package ● Camera HAL – The camera module layer implemented by SoC vendors. The app-level public frameworks are built on top of the camera HAL ● Camera HAL3.1 – Version of the camera device HAL released with Android 4.4 ● Camera HAL3.2 – Version of the camera device HAL released with Android 5.0
  51. 51. Camera HAL (Hands-on : enabling v4l2) ● $ make ARCH=arm menuconfig ● Select “Device Drivers” ● Select “Multimedia Support” ● Select “V4L platform devices” ● Select “Broadcom BM2835 MMAL camera interface driver” ● Save configuration and exit
  52. 52. Camera HAL (Hands-on : enabling v4l2) ● $ ARCH=arm CROSS_COMPILE=arm-linux-androideabi- make zImage -j4 ● $ ARCH=arm CROSS_COMPILE=arm-linux-androideabi- make dtbs -j4 ● Copy zImage and bcm2710-rpi-3-b.dtb in /boot folder (SD card) ● Copy vc4-kms-v3d.dtbo in /boot/overlay folder (SD card)
  53. 53. Enabling Camera ● Add following parameters in /boot/config.txt – start_x=1 – gpu_mem=256 (128 or more) ● Ensure start_x.elf, fixup_x.dat files are present in /boot folder in SD card
  54. 54. Camera Interface (camera_device_ops_t) ● set_preview_window ● set_callbacks (notification and data) ● enable_msg_type ● disable_msg_type ● msg_type_enabled ● start_preview ● preview_enabled ● store_meta_data_in_buffers
  55. 55. Camera Interface (camera_device_ops_t) ● start_recording ● stop_recording ● recording_enabled ● release_recording_frame ● auto_focus ● cancel_auto_focus ● take_picture ● cancel_picture
  56. 56. Camera Interface (camera_device_ops_t) ● set_parameters ● get_parameters ● put_parameters (to return memory to HAL) ● send_command (to driver) ● release (hardware resources) ● dump (state of camera)
  57. 57. Camera Interface (preview_stream_ops_t) ● deque_buffer ● enqueue_buffer ● cancel_buffer ● set_buffer_count ● set_buffers_geometry ● set_crop ● set_usage ● set_swap ● get_min_undequed_buffer_count ● lock_buffer ● set_timestamp
  58. 58. Video4Linux (V4L2)
  59. 59. What is V4L2? ● An open source standard to capture real time video for linux systems ● Second version of video for linux ● V4L2 is two layer driver system ● Top layer is videodev module ● Lower layer is collection of several driver modules ● V4L2 drivers are clients of videodev
  60. 60. V4L2 (Devices) Device Name Minor Number Description From To /dev/video0 /dev/video63 0-63 Video Capturer Devices /dev/radio0 /dev/radio63 64-127 AM/FM Radio Devices /dev/vtx0 /dev/vtx31 192-223 Teletext Devices /dev/vbi0 /dev/vbi15 224-239 VBI Devices
  61. 61. V4L2 (Video Device) # Members Description char name[32] Name of the device int type Type of V4L2 device int minor Device minor number Pointer *fops File operations used Function pointer void (*release)(struct video_device *vfd) Release function used by the driver Void pointer priv The private data
  62. 62. V4L2 (File operations) # Funcrion Description 1 int (*open)() Called when a file descriptor is opened on the device file (/dev/videoX) 2 int (*close)() Called on last close (release) of a file descriptor 3 int (*read)() Called to data (buffer) of size count (the number of bytes of data requested) 4 int (*write)() Called to write data of size count (the number of bytes of data to write) 5 int (*ioctl)() Called when the application calls ioctl() 6 int (*mmap)() Called when the application calls mmap() 7 int (*poll)() Called when the application calls select()
  63. 63. V4L2 (Property Negotiation) ● First the application asks for the possibles values of some property ● Next chooses one of the possible values ● Next configures that value ● Finally checks right configuration of the value ● Important properties - Video Input, Video Output, Norm (only for input devices), Modulator (only for output devices), Input Channel, Window Size
  64. 64. V4L2 (Pixel format negotiation) ● Pixel format is how every pixel is stored in memory ● Application need to know this format to allow the properly interpretation of that pixel ● There are two "families" of pixel formats RGB and YUV ● Mostly, devices capture natively in YUV formats ● Video is converted to RGB formats for displaying in the viewer
  65. 65. V4l2 (Buffers) ● V4L2_MMAP – Memory mapping (allocated by the kernel) ● V4L2_USERPTR – User memory (allocated by user app) ● DMABUF, read/write – Direct Memory Access (GPU, OpenGL) – Can handle fullHD (1080p) at 60FPS
  66. 66. V4L2 (Driver Compilation) ● Step 1 : Copy driver source files in following folder – kernels/rpi/deriver/media/platform/<device> ● Step 2 : Edit Makefile to add following lines – <driver_name>-obj := (add all file which required to build mofules with “.o” extantion) – obj-$(CONFIG_<unique word which easly identify and give information about driver>) +=<driver_name>.o ● Step 3 : Add following line in Kconfig – connfig <driver_name which show on menuconfig> – tristate “<write the purpose of module>” – default n (Must specify; other options y and m) – --help-- <write brief description about module to help user selection while building the kernel module>
  67. 67. V4L2 (Driver Compilation and loading) ● Step 4 : make ARCH=arm menuconfig – Select driver with m option; save and exit ● Step 5 : Compile module from /kernel/rpi directory – make ARCH=arm CROSSCOMPILE=<absolute path of compiler> -j4 modules ● Step 6 : copy <driver>.ko file in target board – system/lib/modules/<kernel version>/kernel/media/v4l2-core ● Step 7 : Run following command – insmod <driver_name>.ko (to load driver) – rmmod <driver_name> (to remove driver)
  68. 68. Stay connected About us: Emertxe is India’s one of the top IT finishing schools & self learning kits provider. Our primary focus is on Embedded with diversification focus on Java, Oracle and Android areas Emertxe Information Technologies Pvt Ltd No. 83, 1st Floor, Farah Towers, M.G Road, Bangalore Karnataka - 560001 T: +91 809 555 7 333 T: +91 80 4128 9576 E: training@emertxe.com https://www.facebook.com/Emertxe https://twitter.com/EmertxeTweet https://www.slideshare.net/EmertxeSlides
  69. 69. Thank You
  • MashkurahmedBadebade

    Nov. 8, 2021
  • TamTranVan3

    Aug. 30, 2021
  • JaehongLee51

    Mar. 24, 2021
  • chenthinker

    Sep. 16, 2020
  • srikantladi

    Nov. 19, 2019
  • RoshinK5

    Jun. 21, 2019
  • sampath_g

    Apr. 15, 2019
  • VijayakumarKuppusamy

    Feb. 18, 2019
  • JedsadaUdomsed1

    Jan. 15, 2019
  • 2112arun

    Sep. 11, 2018
  • RaghavendraReddyK1

    Aug. 4, 2018
  • HoCheolJeon1

    Jul. 27, 2018
  • ZenLeung

    May. 3, 2018
  • torumodos

    Apr. 19, 2018
  • milapkgajjar

    Jan. 17, 2018
  • miyabi0927

    Nov. 21, 2017
  • levdolgov

    Jul. 12, 2017
  • hemadivakala

    Jun. 15, 2017
  • SHRISHAUDUPA

    May. 30, 2017
  • GiancarloEstcio

    May. 1, 2017

Android Audio HAL – Audio Architecture – Audio HAL interface – Audio Policy – Audio HAL compilation & verification – Overview of Tinyalsa Android Video HAL – Camera Architecture – Overview of camera HAL interface – Overview of V4L2 – Enabling V4l2 in kernel – Camera HAL compilation and verification

Views

Total views

9,860

On Slideshare

0

From embeds

0

Number of embeds

34

Actions

Downloads

436

Shares

0

Comments

0

Likes

20

×