The document discusses contested collective intelligence and sensemaking. It describes how collective intelligence infrastructures can augment human capacity for sensing, responding to, and shaping environments. Specifically, it discusses how tools that detect patterns in documents and allow annotations and linking of interpretations can support sensemaking by facilitating the creation of plausible narratives to explain evidence. It also provides examples of prototype tools like Cohere that allow structured debates and linking of questions, evidence and arguments.

1. PARC, Apr 1st 2011
Contested Collective Intelligence
Resilience, Complexity & Sensemaking
Simon Buckingham Shum & Anna De Liddo
Knowledge Media Institute, Open Learning Network Project
Open University UK
http://people.kmi.open.ac.uk/sbs
http://people.kmi.open.ac.uk/anna
1

2. Acknowledgements
Open Learning Network project (2009-12): olnet.org
funded by the William & Flora Hewlett Foundation
OLnet Collective Intelligence workstream:
http://olnet.org/collective-intelligence
Developing conceptual foundations and infrastructure (people+proceses+tools)
for Contested Collective Intelligence on the open social web.
Example: Open Education Evidence Hub http://ci.olnet.org (alpha)
2

3. Your team, organization, school, professional
network, community...
3

4. Your team, organization, school, professional
network, community...
4

5. Your team, organization, school, professional
network, community...
5

6. How do we augment this system’s capacity to
sense, respond to, and shape its environment?
§ Through the lens of complex
adaptive systems, resilience and
network science...
§ Through the lens of
sensemaking and HCI...
§ Hypermedia Discourse: social-
semantic web + models of
discourse
6

7. How do we augment this system’s capacity to
sense, respond to, and shape its environment?
§ Through the lens of complex
adaptive systems, resilience and
network science...
§ many interacting agents (human
and software)
§ many weak signals that can
build up unexpectedly
§ diversity and redundancy
§ feedback loops
§ visual analytics to reveal
emergent patterns and network
properties
§ ability to withstand change and
shock to the system
7

8. Resilience
§ Walker, et al. (2004) define resilience as
“the capacity of a system to absorb
disturbance and reorganize while
undergoing change, so as to still
retain essentially the same function,
structure, identity, and feedbacks”
8

9. Resilience Platforms
http://www.futureofed.org/driver/Platforms-for-Resilience.aspx
9

10. Resilience Platforms
http://www.futureofed.org/driver/Platforms-for-Resilience.aspx
Creating flexibility and innovation amid system failures
“Platforms for resilience - enabling
responsive flexibility, distributed
collaboration, and transparency - will allow
institutions to meet such challenges
through innovation, adaptation, and
openness.”
10

11. Resilience in knowledge-intensive ecosystems
When knowledge and understanding are
key variables in the system, resilience
depends on the capacity for learning
e.g. awareness of discrepant evidence,
critical practice, reflection and dialogue
when confronted by challenges or shocks
to the system.
11

12. How does this help?
Working hypothesis:
Confronted by overwhelming complexity...
(e.g. incomplete, ambiguous data, complex adaptive systems, diverse
perspectives, technical/social/political dimensions, time pressure…)
…Personal and Collective Cognition
break down in particular ways…
We need Theories, Tools and Practices
in order to create CI for tackling such dilemmas
(and we need ways to teach these, both to our children, and the current workforce)
12

13. Augmenting human intellect (ack. Engelbart)
Phenomenon Role for CI infrastructure?
Dangers of entrained thinking from experts who • Pay particular attention to exceptions
fail to recognise a novel phenomenon • Computer-supported argumentation
• Make the system open to diverse
perspectives ontologically, and in usability
Complex systems only seem to make sense • Stories and coherent pathways are
retrospectively: narrative is an appropriately important
complex form of knowledge sharing and • Reflection and overlaying of interpretation(s)
reflection for such domains is critical
Patterns are emergent • Generate gestalt views from the data
evidenced in the platform, not from
preconceptions
Much of the relevant knowledge is tacit, shared • Scaffold the formation of significant inter-
through discourse, not formal codifications personal, learning relationships
Many small signals can build over time into a • Enable individuals to highlight important
significant force/change events and connections à aggregate
• Recommend connections based on different
kinds of significant relationship
13
Sources include: Weick (1995); Kurtz & Snowden (2003); Browning, L. and Boudès, T. (2005); Hagel et al (2010)

14. Designing CI to embody resilience principles
Resilience principle Role for CI infrastructure?
build in the potential for diversity • manage diversity of worldviews, and the
tensions this sets up
make tight feedback loops • shared awareness of dis/agreement
amongst peers
promote building of trust/social capital • using social media to build learning
particularly for learning and relationships: trust, affirmation, challenge
sensemaking
enable experimentation • effective dissemination of findings in
relation to key issues and what is already
known
use a decentralised, modular • both technically (enabling innovation,
architecture interoperability and mashups) but also in
how we represent interpretations (ideas
as networks, not big chunks of text)
a stable state – however temporary – in • model key coherence relations; explore
epistemic terms is a plausible narrative narrative indexing 14

15. How do we augment this system’s capacity to
sense, respond to, and shape its environment?
§ Through the lens of
sensemaking and HCI...
§ many plausible narratives: what
was, is, or might be going on?... • cri
tical t
§ many representational artifacts • arg hinkin
being shared and annotated ument g
• rhe ation
§ attention to the quality of torica
conversation: how well are • ass l mov
agents listening to each other umpti es
and what kinds of contributions
• ana ons
logica
do they make? • ca u l thin
§ informal interaction mixed with sality king
• jux
stronger public claims taposi
§ many connections being made, • “ki tions
nda r
both formal and fuzzy elated
...” 15

16. Sensemaking: the search for plausible, narrative
connections
§ In their review of sensemaking, Klein, et al. conclude:
§ “By sensemaking, modern researchers seem to
mean something different from creativity,
comprehension, curiosity, mental modeling,
explanation, or situational awareness, although all
these factors or phenomena can be involved in or
related to sensemaking. Sensemaking is a
motivated, continuous effort to understand
connections (which can be among people, places,
and events) in order to anticipate their trajectories
and act effectively. […] A frame functions as a
hypothesis about the connections among data.”
16

17. Sensemaking
Weick proposes that:
§ “Sensemaking is about such things as
placement of items into frameworks,
comprehending, redressing surprise,
constructing meaning, interacting in pursuit of
mutual understanding, and patterning.” (Weick,
[23], p.6)
17

18. Sensemaking
Weick:
§ “The point we want to make here is that
sensemaking is about plausibility, coherence,
and reasonableness. Sensemaking is about
accounts that are socially acceptable and
credible” ([23] p.61)
18

19. contested collective intelligence...
conversations are critical to sensemaking
there is no master worldview
we need CI infrastructures to pool
awareness of how people are reading
small signals, and amplify important
connections
19

20. Where our tools fit… Given a wealth of
documents…
20

21. Where our tools fit… Given a wealth of
documents, and tools to detect and render
potentially significant patterns…
21

22. Where our tools fit… Given a wealth of
documents, and tools to detect and render
potentially significant patterns…
22

23. Where our tools fit: making meaningful
connections between information elements…
23

24. Where our tools fit: making meaningful
connections between interpretations
interpretation
interpretation
interpretation
interpretation
24

25. Where our tools fit: making meaningful
connections between interpretations
interpretation
interpretation interpretation
(a hunch – no
grounding
evidence yet)
interpretation interpretation
interpretation
25

26. Where our tools fit: making meaningful
connections between information elements
interpretation
Is pre-requisite for
interpretation interpretation
(a hunch – no
grounding
evidence yet)
causes predicts
interpretation interpretation
interpretation
26

27. Where our tools fit: making meaningful
connections between information elements
interpretation
Is pre-requisite for
prevents
interpretation interpretation
(a hunch – no
grounding Is inconsistent with
evidence yet)
causes predicts
challenges
interpretation interpretation
interpretation
27

28. Where our tools fit: building the story that makes
sense of the evidence… i.e. plausible arguments
Question
responds to
motivates
Answer
Assumption
supports challenges
Supporting Challenging
Argument…
Argument…
28

29. Where our tools fit: building the story that makes
sense of the evidence… i.e. plausible arguments
Question
responds to
motivates
Answer
Hunch
supports challenges
Supporting Challenging
Argument…
Argument…
29

30. Where our tools fit: building the story that makes
sense of the evidence… i.e. plausible arguments
Question
responds to
motivates
Answer
Data
supports challenges
Supporting Challenging
Argument…
Argument…
30

31. a prototype infrastructure for
collective intelligence/social learning
http://cohere.open.ac.uk
Convergence of…
web annotation
social bookmarking
concept mapping
structured debate 31

32. Structured deliberation and debate in which
Questions, Evidence and Connections are
first class entities (linkable, addressable, embeddable, contestable…)
32

33. Structured deliberation and debate in which
Questions, Evidence and Connections are
first class entities (linkable, addressable, embeddable, contestable…)
33

34. — web annotation of OER (Firefox extension)

35. User/community-defined visual language
35

36. Structured deliberation and debate in which
Questions, Evidence and Connections are
first class entities (linkable, addressable, embeddable, contestable…)
36

37. Concept Social
Network Network
Social
Discourse
Network

38. Cohere analytics
By looking at
the post type
table it is
possible to
evaluate
learner’s
performance
connecting the
discourse
outcomes with
the specific
learning goal.

39. Cohere analytics
Legend:
Neutral link type
Positive link type
Negative link type

40. Comparing usage of connections

41. Comparison of one’s own ideas to others
Does the learner compare his/her own
ideas to that of peers, and if so, in
what ways?
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-Centric
Learning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff

42. Link broker: connecting other people’s ideas
Does the learner act as a broker,
connecting the ideas of his/her
peers, and if so, in what ways?
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-Centric
Learning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff

43. seeing the connections people make as
they annotate the web using Cohere
Visualizing all the connections that a
set of analysts have made
— but unfiltered, this may not be very
helpful

44. — semantic filtering of connections
Visualizing multiple learners’
interpretations of global
warming sources
Connections have been filtered
by a set of semantic
relationships grouped as
Consistency
De Liddo, A. and Buckingham Shum, S. (2010). Cohere: A prototype for contested collective intelligence. In: ACM Computer Supported Cooperative Work
(CSCW 2010) - Workshop: Collective Intelligence In Organizations, February 6-10, 2010, Savannah, Georgia, USA. http://oro.open.ac.uk/19554

45. — web annotation for sensemaking

46. OLnet
is
searching
out
the
evidence
for
http://www.flickr.com/photos/bartelomeus/4184705426/
effec4ve
OER,
and
building
an
Evidence
Hub
—
a
living
map
by,
of
and
for
the
OER
movement
—
and
those
we
need
to
impact

47. Moving from document annotation to
connection-making for sensemaking
!

48. Discourse analysis with Xerox Incremental Parser
Detection of salient sentences based on rhetorical markers:
BACKGROUND KNOWLEDGE: NOVELTY: OPEN QUESTION:
Recent studies indicate … ... new insights provide direct … little is known …
evidence ... … role … has been elusive
… the previously proposed …
... we suggest a new ... approach ... Current data is insufficient …
… is universally accepted ...
... results define a novel role ...
CONRASTING IDEAS: SIGNIFICANCE: SUMMARIZING:
… unorthodox view resolves … studies ... have provided The goal of this study ...
paradoxes … important advances Here, we show ...
In contrast with previous Knowledge ... is crucial for ... Altogether, our results ...
hypotheses ... understanding indicate
... inconsistent with past valuable information ... from
findings ... studies
GENERALIZING: SURPRISE:
... emerging as a promising We have recently observed ...
approach surprisingly
Our understanding ... has grown We have identified ... unusual Ágnes Sándor & OLnet Project:
http://olnet.org/node/512
exponentially ... The recent discovery ... suggests
... growing recognition of the intriguing roles
importance ...

49. XIP annotation to Cohere

50. XIP annotations in Cohere’s Firefox Ideas sidebar

51. XIP annotations in Cohere’s Firefox
Connections sidebar

52. XIP annotations visualized in Cohere (ack: prefuse)

53. XIP/Cohere integration: conclusions from
analysis of the corpus (ack: Ágnes Sándor, XRCE)
§ Machine annotation can effectively draw attention to
key issues and contrasting ideas, in a cost effective
and timely manner
§ Human annotation adds higher-level cognitive
activities such as abstracting, contextualizing and
summarizing.
An appropriate combination of both machine and
human annotation can augment and enhance both
human and machine analysis.

54. OpenEd Evidence Hub: ci.olnet.org
an alpha release
54

55. OpenEd Evidence Hub: ci.olnet.org
an alpha release
55

56. OpenEd Evidence Hub: ci.olnet.org
an alpha release
56

57. olnet.org
ci.olnet.org
57