Wednesday Nights in the Tetherless World (TWed), November 28th, 2012

1. Joshua Shinavier
Semantics and Sensors
Wednesday Nights in the Tetherless World (TWed)
November 28th, 2012

2. Overview
• intro to sensors
• semantic sensor networks
• components of an application
• a demo
2

3. What is a sensor?
• any device that measures a physical quantity
• e.g. temperature, pressure, electrical potential
• more abstract quantities are sometimes measured by
“sensors”
• e.g. the number of bicycles available at a bicycle
sharing station
• however, for our purposes...
3

4. Sensors look like this
4

5. How do you talk to a sensor?
• basic sensors produce simple analog or digital
signals,
• or use simple protocols such as I²C
• they don’t understand HTTP
• nor even TCP/IP
• they need to be connected with a more general-
purpose device
• e.g. a microcontroller
5

6. OK, how do you talk to a microcontroller?
• concerns of low-power, low-resource devices
• constrained CPU and memory
• battery life, if battery-powered
• range, if wireless
• there are many solutions
• e.g. Constrained Application Protocol (CoaP)
• interoperates with HTTP
• here, we will use Open Sound Control (OSC)
• simple messaging format for multimedia devices
6

7. Some interrelated concepts
• ubiquitous computing
• ambient intelligence
• perceptual intelligence
• Humanistic Intelligence
• wearable computers
• Internet of Things
• Web of Things
7

8. Some quantities we can measure with
sensors
8

9. Simple physics
• force / pressure
• electrical potential,
• strain / deformation current
• flex / bend • magnetic field strength
• stretch • flow of liquids and
gases
• linear displacement
• angular displacement
• etc. etc. etc.
9

10. The environment
• air temperature
• barometric pressure • ambient light level(s)
• humidity • ambient noise
• dust and smoke level • ambient vibration
• gas concentrations
10

11. Where you are, what’s around you
• GPS position • identified objects
• compass heading • (RFID, visual object
recognition, etc.)
• attitude, acceleration,
• distance and motion
angular momentum
• altitude • (doppler,
rangefinders)
11

12. Sensing yourself
• respiratory rate • muscle activity (EMG)
• heart rate / pulse • eye movement (EOG)
• blood pressure • brain activity (EEG)
• skin temperature • body position and
movement
• skin resistance and
capacitance • facial expression...
• perspiration • voice...
12

13. Common themes in semantic sensor
networks
13

14. Sources of sensor data and metadata
• gathering sensor data from source x, e.g.
• sensor data from wireless networks
• sensor data from mobile phones
• citizen sensing
• gathering sensor metadata
• through human input
• derived from raw measurements
14

15. Modeling and publication of sensor data
• ontological representation of sensor data
• sensor metadata as Linked Data
• integration of streaming sensor data with static
metadata, web service APIs
• annotation of sensor data with quality measures
• publishing historical time series data
15

16. Applications of sensor data
• complex event processing over sensor data streams
• composition of sensor web APIs
• ubiquitous computing applications
• search over sensor data
• stream reasoning over sensor data
16

17. Components of a semantic sensor
network application
17

18. Two kinds of sensor-based applications
• time-series applications
• sensors make observations at regular intervals
• e.g. air temperature sampled every minute, smoke
level sampled every ten seconds
• event-driven applications
• sensors make observations in response to events
• e.g. movement detected via infrared or RFID,
flipping of a switch, events of voice or gesture
18

19. Sensor network ontologies
• Observations and Measurements model in OWL
• OGC Sensor Web Enablement Initiative
• Semantic Sensor Network ontology
• W3C Semantic Sensor Network Incubator Group
• OntoSensor ontology
• builds on SUMO and ISO 19115 (geo metadata)
• there are many others
19

20. Modeling an observation
Sensor Place
Feature
location
featureOfInterest
Observation
observedProperty
Property
samplingTime result
Interval ResultData
hasEnd value uom
hasBeginning
Instant Instant (numeric) UnitOfMeasurement
20

21. Some options for RDF streams
• SPARQL 1.1 Update
• SparqlPuSH
• SPARQL + pubsubhubbub
• RDFAgents
• provenance chains via Named Graphs
• roll your own
21

22. Continuous SPARQL
• typically, SPARQL is used to query for historical data
• continuous SPARQL queries for future data
• first submit the query, then wait for matching data
• as matching data arrives, answers are produced
• continuous SPARQL languages and frameworks:
• C-SPARQL, SPARQLstream, EP-SPARQL,
StreamingSPARQL, Groppe et al.’s, INSTANS, ...
22

23. Demo: the Monitron
humidity and
temperature
sensor
barometric optical dust
pressure sensor detector
RGB
LED
Omnisensory host
Monitron computer
speaker
electret
microphone PIR motion
color light sensor
sensor
piezo vibration
sensor photoresistor
23

24. Monitron architecture
• Arduino Nano + multiple sensors
• sensors are sampled continuously in a cycle
• samples are aggregated, observations made at
regular intervals
• observations and error messages are sent to a host
computer as OSC messages
• host computer may send commands to the Monitron
• a process on the host computer rdfizes received
observations,
• adding them to an RDF stream using RDFAgents
• each observation is contained in an RDF graph
24

25. Step #1: build the device
• order parts
• snip wires
• solder here
• heat-shrink there
• done!
25

26. Step #2: write the firmware
• the output of the device looks like:
/om/system/time 1753A23
/om/sensor/7bb206l0/vibr 1753A23 1754CEE 10000 0.003 0.022 0.021 0.000000
/om/sensor/md9745apzf/sound 1753A23 1754CEE 10000 0.254 0.792 0.507 0.001740
/om/sensor/photo/light 1753A23 1754CEE 10000 0.561 0.567 0.565 0.000020
/om/sensor/rht03/humid 1754CFC 1754D03 1 0.490 0.490 0.490 0.000000
/om/sensor/rht03/temp 1754CFC 1754D03 1 1.000 22.600 22.600 0.000000
/om/sensor/bmp085/press 1754D0B 1754D17 1 1.000 101209.000 101209.000 0.000000
/om/sensor/bmp085/temp 1754D0B 1754D17 1 1.000 23.500 23.500 0.000000
/om/sensor/gp2y1010au0f/dust 1754D1F 1754D29 1 0.160 0.160 0.160 0.000000
/om/sensor/se10/motion 1753A23 1754D2E 1 0
• for each observation:
• sensor id, time interval, #samples, min val, max
val, mean, variance
26

27. Step #3: extend the sensor ontology
• you need classes for the
specific sensors of your device
• each one produces a different
kind of observation
• with different units of
measurement
27

28. Step #4: describe your sensors
• each sensor is an instance of a Sensor subclass
• a device contains more than one sensor if it measures
more than one quantity (e.g. temperature + pressure)
:avago-adjd-s311-cr999_1 a m:ColorLightLevelSensor ;
rdfs:label "color light level sensor of Monitron #1" ;
rdfs:comment "an Avago Technologies ADJD-S311-CR999 miniature surface mount RGB digital
color sensor which has been calibrated to output red, green, and blue color values from 0 to 1" .
:bosch-bmp085_1_thermometer a m:Thermometer ;
rdfs:label "thermometer #1 of Monitron #1" ;
rdfs:comment "a Bosch BMP085 digital (barometric) pressure sensor which has been calibrated
to output temperature values in degrees Celsius" .
:bosch-bmp085_1_barometer a m:Barometer ;
rdfs:label "barometer of Monitron #1" ;
rdfs:comment "a Bosch BMP085 digital (barometric) pressure sensor which has been calibrated
to output pressure values in pascals" .
28

29. Step #5: add programming logic
• what should the host computer do when it receives a
“coffeepot half full” observation?
• each type of observation must be translated in a
slightly different way
• a class hierarchy in Java works nicely
29

30. Step #6: start streaming
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix monitron: <http://fortytwo.net/2012/08/monitron#> .
@prefix om: <http://schemas.opengis.net/om/2.0/> .
@prefix time: <http://www.w3.org/2006/time#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix : <http://fortytwo.net/2012/08/universe#> .
:event-UXhu5B5
om:Procedure :bosch-bmp085_1_thermometer ;
om:featureOfInterest :room_1 ;
om:observationLocation :room_1 ;
om:observedProperty monitron:airTemperature ;
om:result [
om:uom monitron:degreesCelsius ;
om:value 23.3 ;
a om:ResultData] ;
om:samplingTime [
a time:Interval ;
time:hasBeginning [
a time:Instant ;
time:inXSDDateTime "2012-11-28T07:50:06.132-05:00"^^xsd:dateTime] ;
time:hasEnd [
a time:Instant ;
time:inXSDDateTime "2012-11-28T07:50:06.145-05:00"^^xsd:dateTime]] ;
a monitron:AirTemperatureObservation .
30

31. Step #7: submit a continuous query
PREFIX : <http://fortytwo.net/2012/08/universe#>
PREFIX m: <http://fortytwo.net/2012/08/monitron#>
PREFIX om: <http://schemas.opengis.net/om/2.0/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-
ns#>
SELECT ?result ?value WHERE {
?obs rdf:type m:VibrationLevelObservation .
?obs om:result ?result .
?result om:value ?value .
?obs om:observationLocation ?loc .
?loc m:containedIn :winslowBuilding .
}
31

32. Step #8: receive results
subscribers
RDFAgents
host
computer
OSC
Monitron
I²C,
etc.
sensors
32

33. Thanks!
• Tetherless World Constellation:
• http://tw.rpi.edu
• Code:
• https://github.com/joshsh/laboratory
• Contact:
• josh@fortytwo.net, @joshsh
33