When developing user interfaces for interacting with data and content one typically assumes that one knows the type of data and one knows how to interact with such type of data. The core idea of the Semantic Web is that data is self-describing, which implies that its semantics is not designed and described at an initial point in time, but it rather emerges by its use. This flexibility is one of the greatest assets of the Semantic Web, but it also severely handicaps intelligent interaction with its data. In this talk, we will sketch the principal problem as well as first steps to deal with the problem of interacting with the unknown.

1. Steffen Staab The Semantic Web: Interacting with the Unknown 1Institute for Web Science and Technologies · University of Koblenz-Landau, Germany
Web and Internet Science Group · ECS · University of Southampton, UK &
The Semantic Web:
Interacting with the Unknown
Steffen Staab
University of Southampton
&
Universität Koblenz-Landau

2. Steffen Staab The Semantic Web: Interacting with the Unknown 2
Daten – Menschen
Meaning?

3. Steffen Staab The Semantic Web: Interacting with the Unknown 3
Traditional Information System
Business
Logics
Structured Data
Unstructured
Data
Presentation and
Interaction
Charakteristics:
• Processes known
• Data structures
known
• Meaning of data in
schema and
implicit in code

4. Steffen Staab The Semantic Web: Interacting with the Unknown 4
Information ecosystems nowadays
Examples
• Open Data
• 1000s
DBs/company
• Ad-hoc data
Characteristics
• Some structure
• Late structure
• Social context
• Meaning of data
most important

5. Steffen Staab The Semantic Web: Interacting with the Unknown 5
How does data receive meaning?
Explicit:
• Formal schema/ontology
– By someone else?
Implicit:
• Names are just used for
describing
Social:
• Communities converge
– By discussion
– By emergence
Meaning?

6. Steffen Staab The Semantic Web: Interacting with the Unknown 6
A Bit like Crisps ...
Sub languages
• For consumer
– title,...
• Global retailers
– barcode
• US food industry
– serving size, calories,...
• Producer
– batch number
...
https://www.youtube.com/watch?v=ga1aSJXCFe0
Depending on who you are – you encounter the
(un)expected, the (un)known, the (un)understandable,...

7. Steffen Staab The Semantic Web: Interacting with the Unknown 7
What is the Semantic
Web/Linked Data?

8. Steffen Staab The Semantic Web: Interacting with the Unknown 8
What is a triple?
http://dbtune.org/musicbrainz/resource/artist/d87e52c5-bb8d-4da8-b941-9f4928627dc8
ABBA
foaf:name

9. Steffen Staab The Semantic Web: Interacting with the Unknown 9
What is Linked Data? The LOD Cloud

10. Steffen Staab The Semantic Web: Interacting with the Unknown 10
What is Linked Data? Linked Data Principles1. URIs as
identifiers
2. http
lookup
3. RDF
(triples)
4. relations, also
to other locations

11. Steffen Staab The Semantic Web: Interacting with the Unknown 11
What is Linked Data good for?
• Data integration is (relatively) easy
– Migrating different data sources to
linked data is (relatively) easy
• Late schema is easy
– Just add some more fields
• Ignoring data is easy
– Think of crisps
• Serendipitous use
– Discover new information & new sources
by following links
• Data repurposing / pointing
– Use what others have done at both schema
and data level

12. Steffen Staab The Semantic Web: Interacting with the Unknown 12
Issue: From Unknown Data Publishing to
Unknown Data (Schema) Understanding

13. Steffen Staab The Semantic Web: Interacting with the Unknown 13
State-of-the-Art: One App at a Time
Shameless self-promotion: Semaplorer
[Schenk et al., JoWS 2009]
Billion Triples Challenge 1. Prize
2008

14. Steffen Staab The Semantic Web: Interacting with the Unknown 14
State-of-the-Art: One App at a Time
Shameless self-promotion: LISA
1. Prize
German
Linked Open Gov Data
Competition 2012

15. Steffen Staab The Semantic Web: Interacting with the Unknown 15
LENA – A Fresnel application
Fresnel Vocab by [Pietriga et al. ISWC-2006]

16. Steffen Staab The Semantic Web: Interacting with the Unknown 16
What‘s between the two?
One App at a Time
+ Great to use
+ Like DB application
- Brittle
- Not really extensible
Generic Frameworks
+ Can be applied on all data
- Data remains hard to
understand
- No process support
- Noone wants to use them

17. Steffen Staab The Semantic Web: Interacting with the Unknown 17
Facet Navigation in an Open
World

18. Steffen Staab The Semantic Web: Interacting with the Unknown 18
Mobile Facets: Exploring Spaces & Events
(Linked) Data Sources
• Places and names:
– Wikipedia
– GeoNames
• Events
– Eventful
– Upcoming
• Photos
– Flickr
Characteristics:
• open list
• unknown categories
and items
• low quality
categorization
• mobile app

19. Steffen Staab The Semantic Web: Interacting with the Unknown 19
Mobile Facets: Exploring Spaces & Events

20. Steffen Staab The Semantic Web: Interacting with the Unknown 20
Conceptual
Navigation Model

21. Steffen Staab The Semantic Web: Interacting with the Unknown 21
Hypertextual Navigation (from D. Schwabe)
What does it mean to
click here?
Semantics is clear,
but Pragmatics?
Context +
Grice‘s Pertinence!
[Bomfim & Schwabe, 2011]

22. Steffen Staab The Semantic Web: Interacting with the Unknown 22
Hypertextual Navigation (from D. Schwabe)
• Input: LOD + Navigation Model + other stuff
• Navigation Model
– A Context is a set of resources that share similar
navigation opportunities.
• Context:Navigation ⇔Class:Structure+Behavior
– Navigation Metamodel

23. Steffen Staab The Semantic Web: Interacting with the Unknown 23
Hypertextual Navigation (from D. Schwabe)
foaf:Document
DocumentsAlpha
byPerson
foaf:Person
PersonsAlpha
byDocument
AllDocuments
AllPersons
DocumentsByPerson
AllEvents
swc:AcademicEvent
EventsAlpha
byUserProgramEventsByUserProgram
byOrganiza on
Hitting a link of type
„Organization“
means different
things in different
contexts!
Here is one!

24. Steffen Staab The Semantic Web: Interacting with the Unknown 24
Programming with the
Unknown

25. Steffen Staab The Semantic Web: Interacting with the Unknown 25
Programming with Linked Data

26. Steffen Staab The Semantic Web: Interacting with the Unknown 26
Programming with Linked Data
Tasks of the Programmer
1 Schema exploration
2 Programming
code types
3 Programming queries
4 Programming procedures
for
• creating,
• manipulating,
• persisting
objects
c1

27. Steffen Staab The Semantic Web: Interacting with the Unknown 27
Node Path Query Language with auto completion
Exploration of classes
Exploration of relations
Exploration of instances

28. Steffen Staab The Semantic Web: Interacting with the Unknown 28
LITEQ in Action

29. Steffen Staab The Semantic Web: Interacting with the Unknown 29
LITEQ in Action

30. Steffen Staab The Semantic Web: Interacting with the Unknown 30
NPQL and LITEQ
NPQL (Node Path Query Language)
• Intensional Queries
 describing classes and relations
• Extensional Queries
 data items
LITEQ (Language Integrated Types, Extensions and Queries)
• Implementation of NPQL as F# Type Provider in Visual Studio
• Auto completion using NPQL queries
• Automatic typing
of extensional query results
by intensional queries

31. Steffen Staab The Semantic Web: Interacting with the Unknown 31
Outlook: Programming with Linked Data
• More expressive query languages
– Derived data types in tractable description logics!
• More precise combined type inference
– (derived) type from data source
– type inference in programming language
• Programming across data sources
– Federated queries
– Linktraversal-based queries (the unknown sources)
• Integration of schema induction
– Low quality of schema/ontologies
• Improved autocompletion

32. Steffen Staab The Semantic Web: Interacting with the Unknown 32
Tags & Eye Tracking
Deriving Semantics from Interaction

33. Steffen Staab The Semantic Web: Interacting with the Unknown 33

34. Steffen Staab The Semantic Web: Interacting with the Unknown 34
Meaning from Cues + Observation
=
car + +
carcar

35. Steffen Staab The Semantic Web: Interacting with the Unknown 35
Pragmatics
A theory for interacting with the unknown?

36. Steffen Staab The Semantic Web: Interacting with the Unknown 36
Austin: How to do things with words
Core Hypothesis by Austin:
Speech is not only passively describing a given reality, but it can
change the (social) reality it is describing through speech acts
Summary from Wikipedia, 2012-06-09

37. Steffen Staab The Semantic Web: Interacting with the Unknown 37
Why Do We Understand the Text Web?

38. Steffen Staab The Semantic Web: Interacting with the Unknown 38
Austin: How to do things with words
• Phonic act / graphic act:
PopulationIs486,855.
• Locutionary act:
population(Duisburg,486,855)
Grice‘s
maxims

39. Steffen Staab The Semantic Web: Interacting with the Unknown 39
Grice‘s Maxims / Cooperative Principle
Interacting agents mutually assume that:
• Quantity:
– Be as informative as you possibly can,
– give as much information as needed, not more.
• Quality:
– Be truthful
• Pertinence:
– be relevant,
– say things pertinent to the discussion
• Manner:
– be clear, brief, orderly as one can
– avoid obscurity and ambiguity
Criteria are
competing and
overlapping.

40. Steffen Staab The Semantic Web: Interacting with the Unknown 40
Austin: How to do things with words
• Phonic act / graphic act:
PopulationIs486,855.
• Locutionary act:
population(Duisburg, 486,855)
• Illocutionary act:
Inform reader about core characteristic of a city
• Perlocutionary act:
Plan for how to travel, use of taxi/rental car…
Grice‘s
maxims

41. Steffen Staab The Semantic Web: Interacting with the Unknown 41
Austin: How to do things with words
Core Hypothesis by Austin:
Speech is not only passively describing a given reality, but it can
change the (social) reality it is describing through speech acts
Summary from Wikipedia, 2012-06-09
Hypothesis of this talk:
Linked data is facts, but the idea of linked data is also
re-purposing,
⟹ re-presenting,
⟹ re-narrating,
to achieve an understandable dialogue

42. Steffen Staab The Semantic Web: Interacting with the Unknown 42
Data is not Text
Quantity:
• One triple vs big data
➯ What is the right amount?
Pertinence:
• Pertinence to dialogue
➯ Does the discussion/interaction determine data
selection?
Manner:
• Data is not sequential
➯ No implicit ordering contained in the data
(e.g. birthdata before date of death)

43. Steffen Staab The Semantic Web: Interacting with the Unknown 43
Semantic Web / Linked Data
• Phonic act / graphic act:
ThetemperatureinMilwaukeeis100°F.
- various syntaxes -
• Locutionary act:
population(Duisburg, 486,855)
- RDF/OWL interpretation –
Generic
applications easily
violate Grice‘s
maxims!
 Quantity
 Quality
 Pertinence
 Manner
Lead question:
Does the Semantic Web
have a pragmatics layer?
How would this look like?

44. Steffen Staab The Semantic Web: Interacting with the Unknown 44
Conclusion

45. Steffen Staab The Semantic Web: Interacting with the Unknown 45
Syntax
Semantics
Pragmatics
Metamodels
Patterns
Rankings
...

46. Steffen Staab The Semantic Web: Interacting with the Unknown 46
Issue: From Unknown Data Publishing to
Unknown Data Understanding
Cognition
Storytelling
Pragmatics
Ontology Patterns
Conceptual Modeling
Metamodels
...
Quantity
Pertinence
Manner

47. Steffen Staab The Semantic Web: Interacting with the Unknown 47
• Size of „unknown“ data is quickly growing
• Interaction with „unknown“ data is underexplored
• Interaction with the Semantic Web is both
– Challenge
– Opportunity
• Example topics/challenges:
– What are best Meta-frameworks?
– What are best styles of interaction for user-defined
semantics of data?
– Conceptual Programming by Non-Programmers
Call-to-arms

48. Steffen Staab The Semantic Web: Interacting with the Unknown 48
Semantic
Web
Social Web &
Web Retrieval
Interactive Web &
Human Computing
Web &
Economy
Software &
Services
Computational
Social Science
Thank You!

49. Steffen Staab The Semantic Web: Interacting with the Unknown 49
Literature
J. L. Austin. How to do things with words. Oxford University Press, 1962/1975.
M. H. de S. Bomfim, Daniel Schwabe. Design and Implementation of Linked Data Applications
Using SHDM and Synth. Int. Conf. Web Engineering 2011, pp. 121-136.
Chierchia, Gennaro.; McConnell-Ginet, Sally: Meaning and Grammar : An Introduction to
Semantics. MIT Press, 1990.
E. Pietriga, C. Bizer, D. Karger, R. Lee: Fresnel: A Browser-Independent Presentation Vocabulary
for RDF. International Semantic Web Conference 2006, Springer 158-171.
S. Schenk, C. Saathoff, S. Staab, A. Scherp. SemaPlorer – Interactive Semantic Exploration of
Data and Media based on a Federated Cloud Infrastructure. In Journal of Web Semantics.
Special issue on Semantic Web Challenge Winners 2008. Elsevier, 7(4), 2009.
Kleinen, A. Scherp, S. Staab. Interactive faceted search and exploration of open social media
data on a touchscreen mobile phone. Multimedia Tools and Applications. DOI 10.1007s11042-
013-1366-3
M. Konrath, T. Gottron, S. Staab, A. Scherp. SchemEX – Efficient Construction of a Data
Catalogue by Stream-based Indexing of Linked Data. In: Journal of Web Semantics. Special
issue on Semantic Web Challenge Winners 2011. Elsevier, Volume 16, November 2012, pp.
52-58.
M. Leinberger, S. Scheglmann, R. Lämmel, S. Staab, M. Thimm, E. Viegas. Semantic Web
Application Development with LITEQ. In: ISWC-2014, LNCS, Springer 2014