Interacting with Intel Edison
with Esther Jun Kim
Presented at FITC Toronto 2015
More info at www.fitc.ca
OVERVIEW
This talk will introduce the audience to Intel Edison, Intel’s newest maker platform. It will show how Edison can be transformed into a smart, natural device by teaching it to see and hear while retaining its small form factor.
The talk will also briefly describe how to interface Edison with simple supplementary hardware to create a programmable, voice-controlled system.
OBJECTIVE
Highlight the possibilities of Speech Interaction with Maker Devices.
TARGET AUDIENCE
Developers, designers, programmers.
FIVE THINGS AUDIENCE MEMBERS WILL LEARN
What’s Intel Edison.
How Language Models Work.
Simple Circuits.
About LEDs.
Voice Interaction.
3. 3
Agenda
Introduction to Edison
Getting Edison to see (with Node.js)
Getting Edison to hear (with Python)
Interfacing with the GPIO / blinking LEDs
Speech recognition / speech-activated LEDs
Additional resources
4. Intel Edison
4
Full-fledged computer in a tiny
package
Dual-core Atom CPU at 500 MHz
MCU at 100 MHz
Wi-Fi / Bluetooth LE
Yocto Linux
40 GPIOs
Software compatibility
C, C++, Python, Node.js support
Intel IoT Developer Kit
6. SparkFun blocks
6
Stackable blocks that expand Edison’s
functionality while retaining its small
form factor
14 blocks so far
Base block – connects peripherals
GPIO block – provides a friendly
interface to the GPIO pins
Base and GPIO blocks
8. 8
High-level overview of video streaming on Edison
Edison (server)
• FFmpeg: encodes the
webcam video to MPEG1
• Node.js: Listens for the
encoded video stream via
HTTP
• WebSockets: broadcasts
the video stream to all
connected browsers
Browser (client)
• jsmpeg: decodes the
MPEG1 video
• HTML5 canvas: used to
render and display the
decoded video
13. Lighting a LED
13
Resistor (330Ω): limits current to the
LED
Switch turns on and off the LED
LED is on when the switch is closed
LED is off when the switch is open
This circuit probably looks very
familiar if you’ve used Arduino
14. Using transistors as
switches
14
SparkFun recommends using
transistors to light LEDs
Transistor basics:
A small amount of current is needed to
use a transistor as a switch
A transistor is controlled by the voltage
at the base pin
Reference: SparkFun
≈
16. Interfacing with the GPIOs using MRAA
16
MRAA is a low level skeleton library for
IO communication on Edison (and
Galileo and other platforms)
It is a C / C++ library that also provides
Python and JavaScript bindings
17. import mraa
# Other libraries we need
import collections
import time
# Map GPIO block pins to MRAA pin numbers
pins = collections.OrderedDict()
pins["GP44"] = 31
pins["GP45"] = 45
pins["GP46"] = 32
pins["GP47"] = 46
# Initialize LED controls
leds = collections.OrderedDict()
leds["G"] = mraa.Gpio(pins["GP44"])
leds["R"] = mraa.Gpio(pins["GP45"])
leds["W"] = mraa.Gpio(pins["GP46"])
leds["Y"] = mraa.Gpio(pins["GP47"])
17
Python
18. # Set direction of LED controls to out
for color in leds:
leds[color].dir(mraa.DIR_OUT)
LED_ON = 1
LED_OFF = 0
while True:
# Turn all LEDs on for 1 second
for color in leds:
leds[color].write(LED_ON)
time.sleep(1)
# Turn all LEDS off for 1 second
for color in leds:
leds[color].write(LED_OFF)
time.sleep(1)
18
Python
21. Speech recognition tools
21
Pocketsphinx: a lightweight version of
CMU Sphinx for natural language
processing
Sphinx Knowledge Base Tool: builds a
set of lexical and language modeling
files for the Pocketsphinx decoder
Corpus
• Set of words / phrases that the
decoder should recognize
• Ex: GREEN, RED, WHITE,
YELLOW, ALL
Modeling files
• Dictionary: Provides a
pronunciation for every word-
like string in a corpus
• Language model: Probabilistic
model that estimates the
likelihood of words / phrases
22. 22
High-level overview of speech-activated LEDs
Record audio
•In 2-sec chunks
•Write .wav file
Decode speech with
Pocketsphinx
•Use the dictionary and language
model files
•Return array of recognized words
Trigger LEDs
•Take appropriate action based on
the recognized words
•Ex: If “RED” in recognized words,
toggle on RED LED