Keynote talk by David De Roure at SSN workshop at ISWC 2012, Boston, 12 November 2012 In many respects the music industry has gone digital "end-to-end", with success stories in Semantic Web adoption. Science too is dealing with a "digital turn" and the R&D community is active with Semantic Web. Meanwhile the Semantic Sensor Network workshop series has demonstrated the applicability of Semantic Web approaches in the sensor network domain. Looking to the future, what can we learn from music and science – and what can they learn from us? In this talk I will draw examples from music and science and introduce a discussion on future work in Semantic Sensor Networks.

1. End-to-End Semantics
Sensors
Semitones
Science
Social Machines
David De Roure

3. Mo7va7on
• This
workshop
series
has
successfully
demonstrated
that
Seman7c
Web
technologies
can
be
used
with
sensor
networks
• It
hasn’t
necessarily
demonstrated
that
they
are
the
technology
of
choice
• What
can
we
learn
from
the
applica7on
of
Seman7c
Web
elsewhere?

4. http://musicnet.mspace.fm/blog/music-linked-data-workshop/

5. To Do
Ingredient List Dissolve 4- Add K2CO3 Heat at reflux Cool and add Heat at Cool and add Extract with Combine organics, Remove Fuse compound to silica &
List
flourinated powder for 1.5 hours Br11OCB reflux until water (30ml) DCM dry over MgSO4 & solvent in column in ether/petrol
Fluorinated biphenyl 0.9 g
Br11OCB 1.59 g biphenyl in completion (3x40ml) filter vacuo
A digital lab book
Potassium Carbonate 2.07 g butanone
Butanone 40 ml
Plan
replacement that
Add Cool
Add Reflux Liquid- Remove Column
Add Reflux Cool Add Dry Filter Fuse
liquid Solvent Chromatography
extraction by Rotary
Evaporation
0.9031 grammes
Weigh
Inorganics dissolve 2
layers. Added brine
~20ml. text
image
3 of 40
Measure
ml
excess
Measure
g
Silica
Ether/
Petrol
Ratio
chemists were able
to use, and liked.
Sample of 4-
Butanone dried via silica column and
Process
measured into 100ml RB flask. flourinated
Record
Used 1ml extra solvent to wash out biphenyl Annotate
container. DCM MgSO4
Annotate
1 1 2 2 1 3 1 4 3 5 2 6 2 7 4 8 9 10 11 12 13 14
Add Cool
Add Reflux Add Remove Column
Add Reflux Cool Liquid- Dry Filter Fuse
liquid (Buchner) Solvent Chromatography
text Sample of
Butanone Annotate
extraction by Rotary
Sample of Br11OCB
Water Annotate Annotate Evaporation
K2CO3
Measure Powder
Weigh Weigh Measure
text
Started reflux at 13.30. (Had to
change heater stirrer) Only reflux
40 text Washed MgSO4 with text
ml for 45min, next step 14:15. Organics are yellow
solution DCM ~ 50ml
2.0719 g g 30 ml
1.5918
Key Observation Types Future Questions
Process weight - grammes Whether to have many subclasses of processes or fewer with annotations
measure - ml, drops Combechem
Input How to depict destructive processes
Jeremy
Frey
annotate - text
Literal 30 January 2004
How to depict taking lots of samples
temperature - K, C ° gvh, hrm, gms
Observation What is the observation/process boundary? e.g. MRI scan

6. Content Navigation throughout
the Content Life-Cycle
• Annota7on
should
occur
within
the
produc7on
process
• Integra7ng
knowledge
of
the
produc7on
workflow
• Managing
and
exposing
this
metadata
using
modern
seman7c
web
and
linked
data
technology
• Empowering
human
producers
and
consumers
semanticmedia.org.uk

7. Some
Social
Machines

8. 1
Music
Sensor
Networks
Science Social
2 3

9. A
Big
Picture
e-infrastructure
Big Data The
Fourth
The Future!
More machines
Big Compute Quadrant
Conventional Social
online
Computation Networking R&D
More people

10. 1
Music
Sensor
Networks
Science Social
2 3

11. http://www.slideshare.net/moustaki/linked-data-on-the-bbc-2638734

12. “Twelve months ago, there were
three of us in the new Olympic
Data Team: … Today, we are a
team of 20, we have built five
applications, provide 174
endpoints, manage 50 message
queues and support ten separate
BBC Olympic products - from the
sport website to the Interactive
Video Player.”
http://www.bbc.co.uk/blogs/bbcinternet/

15. ê
INT. VERSE VERSE BRIDGE VERSE BRIDGE VERSE OUT.
Ichiro
Fujinaga

16. Structural Analysis of Large Amounts of Music Information
23,000 hours of Digital
Music
recorded music
Collec7ons
Music Information
Retrieval Community
Student-­‐sourced
Community
ground
truth
SoLware
Supercomputer
Linked
Data
Repositories

17. Segment
Ontology
class structure
Ontology models properties from musicological domain
• Independent of Music Information Retrieval research and
signal processing foundations
• Maintains an accurate and complete description of
relationships that link them Kevin
Page
and
Ben
Fields

18. Digital
Music
Digital
Music
Digital
Music
Digital
Music
Collec7ons
Collec7ons
Collec7ons
Collec7ons
ground
truth
ground
truth
ground
truth
Community
Community
Community
SoLware
SoLware
Exper7se
Exper7se
SoLware
Exper7se
Exper7se
Results
Results
papers
Results
Results
papers
papers
Evalua7on
Evalua7on
Papers
Infrastructure
Infrastructure
(sociotechnical)
Evalua7ons
Evalua7ons
(sociotechnical)
Evalua7ons

19. story arcs and
timey-wimey stuff
Mike Jewell
Faith Lawrence

20. Music
• Industry
adop7on
(fits
with
prac7ce
and
solves
a
problem?)
• Significant
benefit
of
automa7on
within
the
enterprise
and
for
public
use
• R&D
community
has
created
socio-­‐technical
infrastructure
• Pushing
towards
capture
at
source

21. 1
Music
Sensor
Networks
Science Social
2 3

22. ...the imminent flood of
scientific data expected
from the next generation of
experiments, simulations,
sensors and satellites
Source: CERN, CERN-EX-0712023, http://cdsweb.cern.ch/record/1203203

24. method
data

26. Co-­‐Evolu7on
of
Research
Objects
Packs
ORE
OAI
Workflows
Research
Objects
W3C
PROV
Computa7onal
Research
Objects

28. Notifications and automatic re-runs
Autonomic Self-repair
Curation
New research?
Machines are users too

29. Science
• Systema7c
processing
of
(big)
data
• Par7cular
aWen7on
to
trust,
provenance,
reproducibility
• Assistance
versus
automa7on
– Taylorisa7on,
provisional
outcomes
– Trust
through
authority
or
the
crowd?
• Scholarly
communica7on
supported
by
Seman7c
Web
“social
objects”
– e.g.
Models,
mashups,
narra7ves,
…

30. 1
Music
Sensor
Networks
Science Social
2 3

31. The
Order
of
Social
Machines
Real life is and must be full of all kinds of
social constraint – the very processes
from which society arises. Computers
can help if we use them to create
abstract social machines on the Web:
processes in which the people do the
creative work and the machine does the
administration… The stage is set for an
evolutionary growth of new social
engines. Berners-Lee, Weaving the Web, 1999

32. Building
a
Social
Machine
Virtual World
(Network of
social interactions) Dave
Robertson
Model of social interaction
Design and Participation and
Composition Data supply
Physical
World
(people
and
devices)

34. http://www.bodleian.ox.ac.uk/bodley/library/special/projects/whats-the-score/research-context

35. Social
• Sociotechnical
systems
perspec7ve
– Fundamental
to
sensor
networks?
– Closing
the
loop
needs
applica7ons
and
users
• Ci7zen
sensing
as
social
machine
• Developing
theory
and
prac7ce
– Design
and
construc7on
of
social
machines

36. 1
Music
Sensor
Networks
Science Social
2 3

37. Case
Study
1
• ICT-­‐enabled
manufacturing
(e.g.
a
phone,
car
parts)
• Sensors
throughout
produc7on
process
• Monitoring
en7re
product
lifecycle
–
life
of
objects
(internet
of
things)
• Collec7ng
data
from
discourse
in
collabora7ve
engineering
process
• Informa7on
privacy
• Pursuing
“Crowd
and
cloud”
philosophy
• Big
Data:
seek
to
detect
“signatures”
in
the
data
in
order
to
op7mise
the
manufacturing
process

38. Case
Study
2
• Studio
as
sensor
network
• End
to
end
seman7cs
from
instrument
(sensor)
to
consumer
(almost
to
the
brain…)
• Reproducible,
repurposeable,
reuseable,
remixable
music
• Business
models
and
DRM
• Social
Objects
=
Score,
MP3
file
Research
Object
=
mix
• Interoperability
standards
(analogue
good
too!)
• Dimension
of
performance,
crea7vity
• Already
using
Seman7c
Web!

39. The brain opera technology: New instruments
and gestural sensors for musical interaction
and performance. Journal of New Music
Research, 28(2), 130–149.

40. Provoca7ons
1. Do
we
have
examples
of
end-­‐to-­‐end
seman7cs
in
SSN?
2. How
do
we
share
the
methods
for
processing
sensor
data?
(soLware,
workflows,
automa7on,
…)
– Provenance,
trust,
reproducibility?
(Linked
Science,
Provenance)
3. Are
we
considering
SSN
as
sociotechnical
systems?
– What
are
the
social
objects
in
SSN?
(data?)
– Social
life
of
objects
(bikes)
– Sensors
as
Ci7zens!
S3N
=
Seman7c
Social
Sensor
Nets?
(Ruben)
– What
are
the
social
machines
in
SSN?
4. What
can
we
do
in
concert
with
science
and
music?
– The
studio
as
sensor
network?
Crea7vity?
(ISWC2013
jammers)
5. Can
our
community
build
a
sociotechnical
infrastructure
to
advance
the
work?
– More
than
(one)
ontology
as
social
object?
(SSN
ontology)

41. david.deroure@oerc.ox.ac.uk
www.oerc.ox.ac.uk/people/dder
www.scilogs.com/eresearch
@dder
Slide
credits:
,
Jeremy
Frey,
Chris7ne
Borgman,
Faith
Lawrence
&
Mike
Jewell,
Ichiro
Fujinaga,
Stephen
Downie,
Kevin
Page,
Ben
Fields,
Carole
Goble,
Dave
Robertson
www.myexperiment.org/packs/347

42. Links
• Seman7c
Media
hWp://seman7cmedia.org.uk/
• Music
Informa7on
Retrieval
Evalua7on
eXchange
(MIREX)
hWp://www.music-­‐ir.org/mirex/
• Workflow
Forever
project
(Wf4Ever)
hWp://www.wf4ever-­‐project.org/
• Future
of
Research
Communica7on
(FORCE11)
hWp://force11.org/
• Theory
and
Prac7ce
of
Social
Machines
(SOCIAM)
hWp://sociam.org/
• W3C
SSN
Community
Group
hWp://www.w3.org/community/ssn-­‐cg/

43. • D.
De
Roure,
C.
Goble
and
R.
Stevens.
The
Design
and
Realisa7on
of
the
myExperiment
Virtual
Research
Environment
for
Social
Sharing
of
Workflows
Future
Genera/on
Computer
Systems
25,
pp.
561-­‐567.
• S.
Bechhofer,
I.
Buchan,
D
De
Roure
et
al.
Why
linked
data
is
not
enough
for
scien7sts,
Future
Genera/on
Computer
Systems
• D.
De
Roure,
David
and
C.
Goble,
Anchors
in
ShiHing
Sand:
the
Primacy
of
Method
in
the
Web
of
Data.
WebSci10,
April
26-­‐27th,
2010,
Raleigh,
NC,
US.
• D.
De
Roure,
S.
Bechhofer,
C.
Goble
and
D.
Newman,
Scien7fic
Social
Objects,
1st
Interna/onal
Workshop
on
Social
Object
Networks
(SocialObjects
2011).
• D.
De
Roure,
K.
Belhajjame,
P.
Missier,
P.
et
al
Towards
the
preserva7on
of
scien7fic
workflows.
8th
Interna/onal
Conference
on
Preserva/on
of
Digital
Objects
(iPRES
2011).
• Carole
A.
Goble,
David
De
Roure
and
Sean
Bechhofer
Accelera7ng
scien7sts’
knowledge
turns.
Will
be
available
at
www.springerlink.com
• Khalid
Belhajjame,
Oscar
Corcho,
Daniel
Garijo
et
al
Workflow-­‐Centric
Research
Objects:
First
Class
Ci7zens
in
Scholarly
Discourse,
SePublica2012
at
ESWC2012,
Greece,
May
2012
• Kevin
R.
Page,
Ben
Fields,
David
De
Roure
et
al
Reuse,
Remix,
Repeat:
The
Workflows
of
MIR,
13th
Interna7onal
Society
for
Music
Informa7on
Retrieval
Conference
(ISMIR
2012)
Porto,
Portugal,
October
8th-­‐12th,
2012
• Jun
Zhao,
Jose
Manuel
Gomez-­‐Perezy,
Khalid
Belhajjame
et
al,
Why
Workflows
Break
-­‐
Understanding
and
Comba7ng
Decay
in
Taverna
Workflows,
eScience
2012,
Chicago,
October
2012