An application developer-focused introduction to the underlying sensor system design considerations and their impact on your Android application. Common problems discussed and solved.

1. Designing An Android
Sensor Subsystem
Pitfalls and Considerations
Jen Costillo
jen@rebelbot.com

2. Simple Choices
User Battery
experience performance
7/15/2012 Costillo- OSCON 2012 2

3. Established
or Innovative Product?
Established Innovation-Driven
• Will I be making another • Do I have new sensors
new product in 6 types?
months? • Are features more
• Is the reference design important than release
considered good enough date?
for the application? • Are money and
resources no
problem?
7/15/2012 Costillo- OSCON 2012 3

4. Forsaking Reference Designs
7/15/2012 Costillo- OSCON 2012 4

5. Going On Your Own
• If you make your own, • But…
– You’re on your own – power ↓
– Integration pains – Control code size
– Test time ↑ – Control mechanical
– Gesture testing footprint
becomes a challenge – In-house expertise
– Calibration blues
– Larger mechanical
footprint
7/15/2012 Costillo- OSCON 2012 5

6. Android Universe
Android Application Application
SensorManager
Frameworks
Sensor JNI
Sensor Service Sensor Manager
Libraries
Sensor HAL
Interface Kernel
Driver
Sensor Driver Linux Kernel
Sensor Hub/
Coprocessor Sensors Hardware
7/15/2012 Costillo- OSCON 2012 6

7. Application
Frameworks
Libraries
Linux Kernel
HARDWARE
Hardware
7/15/2012 Costillo- OSCON 2012 7

8. Hardware Architecture
7/15/2012 Costillo- OSCON 2012 8

9. Sampling Rates:
The 3 Rates
Under-sampling Over-sampling
• Inaccurate, sluggish • Accurate, smooth
response response
• Slight power savings • Power-hungry
Sampling
Rate
7/15/2012 Costillo- OSCON 2012 9

10. Wake up events and power
considerations
Application Internal External
Processor only Coprocessor Processor
D C
Reference supported Reference supported More processor selection
Most power hungry Most work done for More outcome control
you Most customized
Footprint impact
7/15/2012 Costillo- OSCON 2012 10

11. Hardware Summary
Power Latency =
Consumption Max(sensorsn)
Sensor
= Σ sensorsn + + dedicated
Solution
any dedicated processing
processor time
• Use tie-breaker criteria
7/15/2012 Costillo- OSCON 2012 11

12. Application
Frameworks
Libraries
Linux Kernel
KERNEL
Hardware
7/15/2012 Costillo- OSCON 2012 12

13. Kernel Driver
Application Processor
Peripheral Shared
Interface Memory
Microcontroller Sensor Coprocessor
7/15/2012 Costillo- OSCON 2012 13

14. Application
Frameworks
Libraries
Linux Kernel
LIBRARIES AND
SERVICES Hardware
7/15/2012 Costillo- OSCON 2012 14

15. Sensor HAL and Services
• HAL
device/<vendor>/<board name>/libsensors
• Service
frameworks/base/services/sensorservice
• Manager
frameworks/base/libs/gui
7/15/2012 Costillo- OSCON 2012
15

16. Sensor Fusion
Libraries
Linux Kernel
Sensor Hub
Sensors
http://en.wikipedia.org/wiki/Sensor_fusion
https://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.html
7/15/2012 Costillo- OSCON 2012 16

17. Gesture Detection Algorithm
Application Android
Processor SensorService
Co-
Sensor Hub Processor
Sensors MPU with
Barometer Proximity
Gyro/Accel
Compass
7/15/2012 Costillo- OSCON 2012 17

18. Gesture Detection Comparison
Make Buy
Application
• Powerful processor
Minimal
Off-load AppPro
Schedule Impact
In-house Already Tested
expertise &tuned
Complete
Compact code
solution Sensor hub
• Off-load to cheaper
power
• Wake up Event
Handling
7/15/2012 Costillo- OSCON 2012 18

19. Calibration
Use of Calibration Gesture in the
Compass App By Catch.com
7/15/2012 19
Costillo- OSCON 2012

20. Application
Frameworks
Libraries
Linux Kernel
FRAMEWORKS
Hardware
7/15/2012 Costillo- OSCON 2012 20

21. Virtual Sensors
• Leverages 1+ physical or other virtual
sensors
• Multiple Options
– Google (version 3)
– Reference Vendor
– Sensor Vendors
7/15/2012 Costillo- OSCON 2012 21

22. Android Virtual Sensors
Accel Gravity
Gravity Linear Accel
Accel
7/15/2012 Costillo- OSCON 2012 22

23. Android Virtual Sensors
Gyro Accel Orientation
Compass Accel Rotation
7/15/2012 Costillo- OSCON 2012 23

24. Virtual Sensors Challenges
Sensor
1
• Garbage In- Garbage Out
• Latency Sensor Virtual
2 Output
• Non-Synchronized
samples Sensor
1
• Implementation Virtual
Dependencies 1
• Multi-vendor problems, Sensor Virtual
2 2
verify Vendor ID
http://www.invensense.com/midc/presentations/James%20Lim.pdf
7/15/2012 Costillo- OSCON 2012 24

25. Application
Frameworks
Libraries
Linux Kernel
APPLICATIONS
Hardware
7/15/2012 Costillo- OSCON 2012 25

26. Using Sensors
mSensorManager =
getSystemService(Context.SENSOR_SERVICE);
mSensorManager.registerListener(
mSensorListener,
mSensorManager.getDefaultSensor(
Sensor.TYPE_ACCELEROMETER),
SensorManager.SENSOR_DELAY_NORMAL);
void onSensorChanged(SensorEvent event) {
// get sensor data
float x =
event.values[SensorManager.DATA_X];
}
7/15/2012 Costillo- OSCON 2012 26

27. Using Sensor (Continued)
SensorManager.getRotationMatrix(
m_rotationMatrix,
null,
m_Mag,
m_Accels);
SensorManager.getOrientation(
m_rotationMatrix,
m_orientation);
float yaw_deg = m_orientation[0] * 57.2957795f;
float pitch_deg = m_orientation[1] * 57.2957795f;
float roll_deg = m_orientation[2] * 57.2957795f;
7/15/2012 Costillo- OSCON 2012 27

28. Types of Sensor Problems
• Bias
• Drift
• Settling Time
• Jitter/Noise
• Environmental
Interference
7/15/2012 Costillo- OSCON 2012 28

29. Bias
• Problem: Data is off
by a constant value.
• Sources:
– static calibration
failure
• Solutions:
– Calculate linear
offset at start of
application
– Recalibrate locally
7/15/2012 Costillo- OSCON 2012 29

30. Drift
• Problem: Shift of
data without cause
• Sources:
– Magnetic
interference
– Poor HW calibration
• Solutions:
– Increase smoothing
techniques
7/15/2012 Costillo- OSCON 2012 30

31. Settling Time
• Problem: Extended
time before
finalized steady
data.
• Sources:
– Latency
– Sensitivity
• Solutions:
– Limit additional
processing
7/15/2012 Costillo- OSCON 2012 31

32. Noise
• Problem: Data jumps
around constantly
• Sources:
– Sensor
– Calibration
– Poor filtering
• Solutions:
– High pass filter
– Linear averaging
– FFT
7/15/2012 Costillo- OSCON 2012 32

33. Environmental Interference
• Problem: Inconsistent
results
• Sources:
– Magnetometer
– EMI
• Solutions:
– Reference Device
– Calibration gesture
7/15/2012 Costillo- OSCON 2012 33

34. Best Practices in
Application Development
• Select the right sensor for the job.
• Use the Correct Data Rate.
– UI or GAMING are the most common.
• Use Sensor In Context
• Customize for your hardware and system
capabilities
• Magnetometer-based sensors are the most
touchy.
• Keep the Gesture UI simple.
7/15/2012 Costillo- OSCON 2012 34

35. QUESTIONS?
JEN@REBELBOT.COM
Additional resources
http://processors.wiki.ti.com/index.php/Android_Sensor_PortingGuide
http://www.kandroid.org/online-pdk/guide/sensors.html
http://invensense.com/midc/
http://developer.android.com/reference/android/hardware/SensorEvent.html
7/15/2012 Costillo- OSCON 2012 35