Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Android beyond the smartphone

1,086 views

Published on

Android is beginning to take off as an embedded operating system alongside Windows Embedded and traditional embedded Linux. What are the motivations for using Android? What are the potential problems? This presentation addresses these points, with use-cases based on real-world applications of Embedded Android

Published in: Software
  • Login to see the comments

Android beyond the smartphone

  1. 1. Android beyond the smartphone Silicon Valley Linux Technology Meetup, 14th September 2016 Android beyond the smartphone 1 Copyright © 2011-2016, 2net Ltd
  2. 2. About Chris Simmonds • Consultant and trainer • Author of Mastering Embedded Linux Programming • Working with embedded Linux since 1999 • Android since 2009 • Speaker at many conferences and workshops "Looking after the Inner Penguin" blog at http://2net.co.uk/ https://uk.linkedin.com/in/chrisdsimmonds/ https://google.com/+chrissimmonds Android beyond the smartphone 2 Copyright © 2011-2016, 2net Ltd
  3. 3. Agenda 1. Embedded Android: 30 minutes Q & A 2. Android BSP: 30 minutes Q & A 3. Interfacing with the real world: 30 minutes Q & A Wrap-up and conclusion Android beyond the smartphone 3 Copyright © 2011-2016, 2net Ltd
  4. 4. Embedded Android Android beyond the smartphone 4 Copyright © 2011-2016, 2net Ltd
  5. 5. Mainstream Android • Android from Google comes in several flavours: • Android, for smartphones and tablets (approx. 1.5 billion of them) • Android Wear, for smart watches • Android TV • Android Auto, for in-vehicle "infotainment" • Plus Brillo, a cut-down Android (no display) for IoT Android beyond the smartphone 5 Copyright © 2011-2016, 2net Ltd
  6. 6. Non-Google Android • Android is an open source operating system • Open source means anyone can join in • Some well-known examples: • Amazon Kindle Fire • BlackBerry Priv • Cyanogen Mod and many other "modder ROMs" Android beyond the smartphone 6 Copyright © 2011-2016, 2net Ltd
  7. 7. Embedded Android • You can create your own custom Android for your own needs • Capitalize on familiar UI • Well-known programming framework • Good development tools Android beyond the smartphone 7 Copyright © 2011-2016, 2net Ltd
  8. 8. Why use Android? • User interface • Attractive • Familiar navigation gestures • Programming framework • Everyone knows Java • Developer tools • Android Studio • Many tools for debugging and profiling • Secure, as a result of SELinux amongst other things • Robust: each app runs in its own sandbox • Updatable: built-in OTA updater Android beyond the smartphone 8 Copyright © 2011-2016, 2net Ltd
  9. 9. Example 1: marine navigation • Furuno NavNet TZtouch2 • Navigation system for small boats • i.MX6 platform • AOSP 5.1 Android beyond the smartphone 9 Copyright © 2011-2016, 2net Ltd
  10. 10. Example 2: point of sale • Leaf Systems • Point of sale terminal Android beyond the smartphone 10 Copyright © 2011-2016, 2net Ltd
  11. 11. How open is Android? • Need to distinguish between the open part: • AOSP • And the proprietary part: • Google Mobile Services • GMS consists of extension libraries that implement the Google APIs, such as maps, in-app payment, Gmail and so on • Let’s take a closer look at the architecture ... Android beyond the smartphone 11 Copyright © 2011-2016, 2net Ltd
  12. 12. Architecture of AOSP Android Java libraries (android.*) Android Java API Applications System services "Glue" libraries Native servers HAL libraries Other libraries Linux Binder JNI Android framework Native layer Kernel App layer Android beyond the smartphone 12 Copyright © 2011-2016, 2net Ltd
  13. 13. The four layers • Applications: stock AOSP apps and 3rd party apps • Framework, consisting of • The android.* libraries, which dispatch messages via binder to ... • ... System services, which use JNI to call down to ... • ... "Glue" libraries written in C++ • Native • The init program, various native servers (daemons) and libraries • Kernel • A Linux kernel customised for the platform Android beyond the smartphone 13 Copyright © 2011-2016, 2net Ltd
  14. 14. Google Mobile Services AOSP Android apps Google Mobile Services Google apps Android beyond the smartphone 14 Copyright © 2011-2016, 2net Ltd
  15. 15. Google Mobile Services • Plain Android apps use only the APIs published within the AOSP • AOSP aps include Launcher2, Browser, Email, Gallery, Keyboard • GMS is a set of services that provide Google proprietary APIs • Most user-facing features require GMS, including popular Google apps such as Play Store, Chrome browser, Google Now, etc. • Requires a license from Google • (You can grab it unofficially (and illegally) via the gapps zip file - google for it) Android beyond the smartphone 15 Copyright © 2011-2016, 2net Ltd
  16. 16. What do you need to run Android? • Hardware from one of the supported architectures • ARM, x86 or MIPS, in 32 or 64 bit varieties • Capable of running Linux • At least 512 MiB RAM • At least 512 MiB flash storage • Touchscreen or external display (optional, but desirable) • GPU with OpenGL ES libraries (optional, but desirable) Android beyond the smartphone 16 Copyright © 2011-2016, 2net Ltd
  17. 17. Dev boards 1/3 Qualcomm DragonBoard 401c • Quad-core ARM Cortex A53 • 1 GiB RAM • Wi-Fi 802.11 b/g/n • Adreno 306 GPU • Expansion headers: 1 x UART, 2 x i2c, 12 x GPIO https://developer.qualcomm.com/hardware/dragonboard-410c Android beyond the smartphone 17 Copyright © 2011-2016, 2net Ltd
  18. 18. Dev boards 2/3 Wandboard • i.MX6 SoC: single, dual or quad core ARM Cortex A-9 • 512 MiB, 1 GiB or 2 GiB RAM • Ethernet and WiFi • Vivante 880 GPU • Expansion headers: 2 x UART, 3 x i2c, 8 x GPIO http://www.wandboard.org Android beyond the smartphone 18 Copyright © 2011-2016, 2net Ltd
  19. 19. Dev boards 3/3 BeagleBone Black • TI AM3359 SoC: single core ARM Cortex A-8 • 512 MiB RAM • Ethernet • Expansion headers: 4 x UART, 2 x i2c, 65 x GPIO https://github.com/csimmonds/android4beagle Android beyond the smartphone 19 Copyright © 2011-2016, 2net Ltd
  20. 20. Questions? Android beyond the smartphone 20 Copyright © 2011-2016, 2net Ltd
  21. 21. Building an Android BSP Android beyond the smartphone 21 Copyright © 2011-2016, 2net Ltd
  22. 22. DIY Android • So, you want to port Android to your platform? Here are the steps: • Make sure it boots Linux (no Android) • Create a simple minimal Android device definition • Boot Android to command shell (no display) • Get ADB USB gadget driver to work • Build Android UI, without GPU support, and get display working • Implement vendor GPU drivers and OpenGL ES libraries • Implement networking, audio, camera, etc as required Android beyond the smartphone 22 Copyright © 2011-2016, 2net Ltd
  23. 23. The Linux kernel • The AOSP repositories do not include a kernel • It is up to you to provide one • Android needs some special kernel features, including: • Android-specific device drivers: binder, logger, anonymous shared memory, etc • Aggressive power management: auto sleep and wake locks Android beyond the smartphone 23 Copyright © 2011-2016, 2net Ltd
  24. 24. Androidisms 1/2 • In mainline Linux since 3.3 • Binder: transaction-based inter-process communication • Anonymous shared memory: allows processes to share memory segments • Low memory killer: kills background applications when memory is low • In mainline Linux since 3.5 • Autosleep/wakelock: suspends CPU unless kernel or app holds a lock Android beyond the smartphone 24 Copyright © 2011-2016, 2net Ltd
  25. 25. Androidisms 2/2 • Still out of tree • Paranoid network security: restricts access to network sockets to apps with android.permission.INTERNET • The recommended kernel configuration options are documented at source.android.com/devices/tech/kernel.html Android beyond the smartphone 25 Copyright © 2011-2016, 2net Ltd
  26. 26. Google reference kernels • Google provide reference implementations for Nexus devices and the emulator at https://android.googlesource.com • There are details on how to build them at http://source.android.com/source/ building-kernels.html • For example, to get the kernel for the emulator ("goldfish"): $ git clone https://android.googlesource.com/kernel/goldfish $ cd goldfish $ git checkout origin/android-3.18 Android beyond the smartphone 26 Copyright © 2011-2016, 2net Ltd
  27. 27. Google kernels: some stats Kernel Latest version Patches(*) Products common 4.1 516 generic exynos 3.4.67 - Nexus 10 goldfish 3.18.0 713 Emulator msm 3.10.73 32968 Nexus 4, 5, 6, 5X, 6P omap 3.10.0 1218 Minnow (Moto 360) samsung 3.0.8 - Nexus S tegra 3.10.40 - Xoom, Nexus 7, 9 x86 3.10.20 - Nexus Player (*) Number of patches from mainline, which you can find like this: $ git remote add stable git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git $ git fetch stable $ git log v<kernel-version>..HEAD $ git format-patch v<kernel-version>..HEAD Android beyond the smartphone 27 Copyright © 2011-2016, 2net Ltd
  28. 28. Vendor kernels • Most vendors base their kernels on the Google reference • ... and then add more patches of their own • Kernel code is licensed under GPLv2, so source is (should be) generally available • Conclusions: • The kernel in most shipping products is quite old • About 500 patches account for Androidisms not in mainline • The remainder are specific to a particular SoC/board/product Android beyond the smartphone 28 Copyright © 2011-2016, 2net Ltd
  29. 29. Getting AOSP • First, you need to get repo: $ curl http://commondatastorage.googleapis.com/git-repo- downloads/repo > ∼/bin/repo $ chmod a+x ∼/bin/repo • Then use it to clone AOSP: $ mkdir ∼/myandroid $ cd myandroid $ repo init -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r55 • The -b option selects the branch • See http://source.android.com/source/ build-numbers.html for comprehensive list Android beyond the smartphone 29 Copyright © 2011-2016, 2net Ltd
  30. 30. Building Android • The Android build system uses GNU make • Controlled by a single top level Makefile plus many fragments: • AndroidProducts.mk: defines products, in sub tree device/ • BoardConfig.mk: defines boards, in sub tree device/ • Android.mk: defines Android packages, throughout AOSP Android beyond the smartphone 30 Copyright © 2011-2016, 2net Ltd
  31. 31. Creating a new device • Device configurations are in the device/ directory • There is a two-level hierarchy • organization/product • The examples that follow use sirius and marvin 1 • Configuration is defined in AndroidProducts.mk, BoardConfig.mk and supporting files • The next few slides describe these essential files... 1In the The Hitchhikers Guide to the Galaxy, Marvin was a paranoid android manufactured by the Sirius Cybernetics Corp Android beyond the smartphone 31 Copyright © 2011-2016, 2net Ltd
  32. 32. AndroidProducts.mk • This is the top-level makefile for the product • Simply references a makefile for each variant of the product • Often there is only one • For example: PRODUCT_MAKEFILES := $(LOCAL_DIR)/aosp_marvin.mk Android beyond the smartphone 32 Copyright © 2011-2016, 2net Ltd
  33. 33. Product makefile example aosp_marvin.mk $(call inherit-product, $(SRC_TARGET_DIR)/product/aosp_base.mk) $(call inherit-product, device/sirius/marvin/device.mk) PRODUCT_NAME := aosp_marvin PRODUCT_DEVICE := marvin PRODUCT_MODEL := Paranoid Android Android beyond the smartphone 33 Copyright © 2011-2016, 2net Ltd
  34. 34. Pre-defined products • In build/target/product: • embedded.mk: minimal build of native layer plus SurfaceFlinger but no Java • aosp_base.mk: embedded.mk, plus Dalvik, Android framework, apps, etc • aosp_base_telephony.mk: aosp_base.mk plus phone dialler and ril • sdk.mk: the SDK • In device/google/atv/products: • atv_base.mk: Android TV Android beyond the smartphone 34 Copyright © 2011-2016, 2net Ltd
  35. 35. Android packages • Android packages (also called Android modules) are units of code that can be compiled separately • Each package is defined by an Android.mk • Name of package given by • LOCAL_PACKAGE_NAME := for Android applications • or • LOCAL_MODULE := for everything else • Package names must be unique: build will fail if not • In Marshmallow there are > 2800 Android.mk files Android beyond the smartphone 35 Copyright © 2011-2016, 2net Ltd
  36. 36. device.mk • device.mk contains additions and overrides to standard configuration • For example, to add packages: PRODUCT_PACKAGES += librs_jni com.android.future.usb.accessory androidvncserver Android beyond the smartphone 36 Copyright © 2011-2016, 2net Ltd
  37. 37. BoardConfig.mk • Defines characteristics of the board • CPU architecture and type • Sizes of file system partitions • Too many settings to document here: look at other BoardConfig.mk files for ideas Android beyond the smartphone 37 Copyright © 2011-2016, 2net Ltd
  38. 38. vendorsetup.sh • lunch searches for vendorsetup.sh in • vendor/*/vendorsetup.sh • vendor/*/*/vendorsetup.sh • device/*/*/vendorsetup.sh • Used to add a product to the menu • Format is add_lunch_combo PRODUCT_NAME-[user | userdebug | eng] • Example add_lunch_combo aosp_marvin-eng Android beyond the smartphone 38 Copyright © 2011-2016, 2net Ltd
  39. 39. Running the build • Select the device $ lunch Lunch menu... pick a combo: 1. aosp_arm-eng [...] 26. aosp_marvin-eng • Start the build $ make -j8 ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=6.0.1 TARGET_PRODUCT=aosp_marvin [...] • And wait several hours Android beyond the smartphone 39 Copyright © 2011-2016, 2net Ltd
  40. 40. The problem with graphics • Android requires OpenGL ES v1.1 to boot • Many applications require OpenGL ES v2 or later • Android provides a fall-back implementation for OpenG LES v1 • A software rendering engine called PixelFlinger • OpenGL ES v2 and later requires hardware, together with kernel drivers and libraries • The problem: drivers and libraries are not open source Android beyond the smartphone 40 Copyright © 2011-2016, 2net Ltd
  41. 41. Living without GPU support • Enable PixelFlinger by adding to kernel command line qemu=1 qemu.gles=0 • Great for board bring-up and early stages of porting • Could be used in a production device with simple UI needs • But, many things need OpenGL ES 2 (e.g. Chrome browser) Android beyond the smartphone 41 Copyright © 2011-2016, 2net Ltd
  42. 42. Questions? Android beyond the smartphone 42 Copyright © 2011-2016, 2net Ltd
  43. 43. Interfacing with the real world Android beyond the smartphone 43 Copyright © 2011-2016, 2net Ltd
  44. 44. Hardware interfacing • Android has support for the hardware you would expect in a smartphone • UI: touchscreen, HDMI monitor, keyboard, mouse, gamepad • Network: mobile data, WiFi, Ethernet • Sensors: accelerometer, magnetometer, ambient light, etc. • But, embedded applications have more diverse needs ... • USB, serial, i2c, digital and analog I/O Android beyond the smartphone 44 Copyright © 2011-2016, 2net Ltd
  45. 45. I/O, securely • In Linux, all I/O is done via file descriptors ("everything is a file") • For security and stability, we want to restrict access to I/O • Android has a complex security model that enforces this Android beyond the smartphone 45 Copyright © 2011-2016, 2net Ltd
  46. 46. The principle of least privilege • Each component (app, system service, native daemon) is only allowed to access resources necessary to do the job • Enforced by file modes and SELinux policy • Package Manager assignes a unique User ID (UID) to each app when it is installed • Apps cannot access any I/O file nodes Android beyond the smartphone 46 Copyright © 2011-2016, 2net Ltd
  47. 47. System services • System resources (display, audio, etc.) are proxied through Android system services running with greater privilege • System services fail the request if app doesn’t have the permission Client app Server process /dev/binder Android beyond the smartphone 47 Copyright © 2011-2016, 2net Ltd
  48. 48. Android permissions • Apps request permissions in AndroidManifset.xml • For example, to access the serial port: <uses-permission android:name="android.permission.SERIAL_PORT"/> • Permission checked by the service • In this case, Serial Manager: SerialService.java enforceCallingOrSelfPermission(android.Manifest.permission.SERIAL_PORT, null); // Throws SecurityException if the client app does not have permission Android beyond the smartphone 48 Copyright © 2011-2016, 2net Ltd
  49. 49. Accessing hardware in Android • There are services for • USB host • USB accessory (described later) • Serial • Other types of interface are not provided Android beyond the smartphone 49 Copyright © 2011-2016, 2net Ltd
  50. 50. USB host mode • Android supports USB connections where Android is the USB host • Applies to most USB devices • Class android.hardware.usb Android beyond the smartphone 50 Copyright © 2011-2016, 2net Ltd
  51. 51. USB accessory mode • Android also has USB "accessory mode" • Android acts as a USB device, the other end acts as the host and provides power • Android Open Accessory (AOA) protocol • AOAv1 from Honeycomb v3.1 (API level 12) • AOAv2 from Jelly Bean v4.1 (API level 16): adds audio from Android to accessessory and HID • Example use: link Android devices to car stereos Android beyond the smartphone 51 Copyright © 2011-2016, 2net Ltd
  52. 52. Serial • Android has (minimal) support for serial ports • Class android.hardware.SerialPort • Requires platform support • Enable serial ports in framework • Permit access to device nodes in SELinux Android beyond the smartphone 52 Copyright © 2011-2016, 2net Ltd
  53. 53. Serial import android.hardware.SerialManager; import android.hardware.SerialPort; [...] private SerialManager mSerialManager; private SerialPort mSerialPort; mSerialManager = (SerialManager)getSystemService(Context.SERIAL_SERVICE); mSerialPort = mSerialManager.openSerialPort("ttyS1", 115200); ret = mSerialPort.read(mInputBuffer); mSerialPort.write(mOutputBuffer, bytes.length); Android beyond the smartphone 53 Copyright © 2011-2016, 2net Ltd
  54. 54. i2c • There are no Android APIs for i2c • However, as a work round: • Modify boot scripts to set mode of /dev/i2c* to 0666 (*) • Either run SELinux in permissive mode or write SELinux policy to allow apps to read/write those nodes • Write a native library that reads/writes the registers on your i2c device (*) There are better ways, but this works. Ideally you would implement an i2c system service and use Android permissions to control access Android beyond the smartphone 54 Copyright © 2011-2016, 2net Ltd
  55. 55. Non-Android code • Use case: porting a middleware layer • Problems • Android C library (Bionic) lacks features -> code won’t compile • Android build system can’t run external Makefiles -> need to re-write as Android.mk files • Android file system layout is not standard -> need to change file paths Android beyond the smartphone 55 Copyright © 2011-2016, 2net Ltd
  56. 56. Separately-compiled binaries • Use an external GNU toolchain, and either: • static link - no library dependencies, but large executables • chroot - create your own root for your program (see later) • Copy the compiled executables and other files into the Android image from device.mk using PRODUCT_COPY_FILES Android beyond the smartphone 56 Copyright © 2011-2016, 2net Ltd
  57. 57. Communication with Android apps • Use standard IPC mechanisms that are supported by Android: • Local (UNIX) Socket (preferred) • Named pipe (also known as FIFO) • A regular file • Can use any libraries, including those not provided by Android • Avoid license restrictions so you can use GPL code • Application and your imported native code run in separate processes, so more robust Android beyond the smartphone 57 Copyright © 2011-2016, 2net Ltd
  58. 58. chroot • Standard utility that runs a command with different root directory • chroot <new root> <command> • The command (and child processes) can only access files "inside" the new root • Often called a chroot jail • See man chroot(8) for details • Android does not include chroot! • Typically, you would use a statically-compiled Busybox which has its own chroot, along with other tools. Android beyond the smartphone 58 Copyright © 2011-2016, 2net Ltd
  59. 59. Creating a chroot • Create a directory containing the files you want • Set the permissions of the socket, (mysocket) to world read/write so that a normal Android application can access it myroot/ |-- bin | |-- myprog |-- etc | |-- my.conf |-- lib | |-- libmylib.so |-- proc | |-- var |-- mysocket Android beyond the smartphone 59 Copyright © 2011-2016, 2net Ltd
  60. 60. What next? • Get a dev board • Start to play • Learn how to build Android from scratch • Begin tinkering with hardware • Keep on learning Android beyond the smartphone 60 Copyright © 2011-2016, 2net Ltd
  61. 61. Classses in Android and Linux • KComputing, in conjunction with 2net, offer a number of 4-day instructor-led classes, including • Android Porting • Android internals • Embedded Linux • System programming for embedded Linux • Open access or on-site • Details at http://2net.co.uk/training.html/ Android beyond the smartphone 61 Copyright © 2011-2016, 2net Ltd
  62. 62. Questions? Slides will be available from the meetup email: training@kcomputing.com (USA) or training@2net.co.uk (rest of world) Android beyond the smartphone 62 Copyright © 2011-2016, 2net Ltd

×