Who to believe? How epistemic cognition can inform science communication Two patients with the same condition decide to research possible treatments. They encounter multiple sources, from experts and others, each with different – sometimes contradictory – information. Depending on whom they believe and how they integrate these claims, the patients may make radically different decisions. These situations are commonplace in everyday life, from medical choices, to our voting decisions. How do we understand these differences, and support people in making the best decisions? Epistemic cognition provides one lens onto this problem. Epistemic cognition is the study of how people think about the justification, source, complexity, and certainty of knowledge. When we evaluate evidence, think about where and when it applies, and connect claims to build models, we engage our epistemic cognition. Understanding how people navigate their own, and others’ knowledge is one of the most pressing social issues of our time in order to develop a sustainable society. I’ll draw on research in epistemic cognition, and my own research on how people search for and talk about evidence, to flag key implications of epistemic cognition research for science communication.

1. Who to Believe
How epistemic cognition can
inform science communication
Dr Simon Knight
@sjgknight
http://sjgknight.com
FacultyofTransdisciplinaryInnovation

2. 2
Two stories…

3. Our protagonists start…
a parent attempting to understand
information around childhood vaccinations;
Public domain image from
Public domain image from
https://commons.wikimedia.org/wiki/File:%22Miracle_Cure!%22
someone seeking to lose weight
investigating the merits of diet versus
regular foodstuffs or supplements.

4. 4

5. 5

6. 6

7. Novices search
‘vertically’. They
find a site and
explore
7
https://tinyurl.com/ASCSearch

8. Novices search
‘vertically’. They
find a site and
explore
8
https://tinyurl.com/ASCSearch
Experts search
laterally to find
out about the
source first

9. 9
https://tinyurl.com/ASCSearch
The twist:
Some people uncritically
navigate many pages.
Some single pages are
authoritative.
How do we help people
corroborate, and identify
credibility?

10. 10
Multiplist Synthesise
& evaluate
Justifies
expertise
Uncritical
(self-affirming?)
adoption

11. 11
It’s not what you know, it’s how
you think about knowledge:
Epistemic cognition

12. “epistemological beliefs are a lens for a learner's views on what is to be learnt”
(Bromme, 2009)
Certainty – static to tentative & evolving
Simplicity – discrete to holistic
Source – external to constructed by self
Justification – authority to evaluation of knowledge (Mason, Boldrin, & Ariasi, 2009)
Epistemic Cognition

13. 13
Multiplist Synthesise &
evaluate
Justifies
expertise
Uncritical
(self-affirming?)
adoption – absolutist view
Who do we trust (and why)?
What do they say; do they agree?
Kuhn, Cheney, & Weinstock, 2000
Evaluativist

14. People who think science is about
absolute truth find uncertain
claims (97% of scientists?) less
convincing; those who think it’s
tentative find it more convincing
14
Image by Sagredo under a CC-By-SA license
People who think that
corroboration and expertise
are important for justification,
consult multiple sources

15. How we process multiple sources & epistemic cognition
(Bråten et al., 2011)
Facet of
cognition
Less adaptive More adaptive
Simplicity Accumulation of facts,
prefer simple sources
Integrated, downplay simple
sources
Certainty Single document
sourcing
Corroboration, represents
complex perspectives and
views showing the diversity of
angles
Source Emphasizes own
opinion, differentiates
between sources less
Emphasizes source
characteristics, distinguishes
between source
trustworthiness
Justification Emphasizes authority,
less corroboration
Emphasizes use of argument
schema and combination of
corroboration and authority

16. 17
These stories matter…

18. People encounter disagreement
through different expert views
So struggle to apply advice to their
demographic and behaviour
This leads to confusion & distrust on
individual topics, which can spread!
21
Why do experts disagree…

19. 23
So what do we do?

20. 24
Knight & Mercer (2015, 2017)
Barzilai,S.,&Ka’adan,I.(2017)
Simple interventions can help:
• Asking people how they’d confirm their knowledge
encourages better source engagement & evaluation
(Porsch & Bromme, 2010).
• Strategic scaffolds help: For each source they
structure, who wrote it, what is their expertise, what are
their interests, when was it published, alongside both
what they said and what evidence they had

21. Get people talking…
Group 1 – Quite successful, characterised: “it’s got all the
important information” “it’s a good site” (quality)
Group 2 – Quite successful, characterised: “I didn’t know
that” (novelty), authority
Group 3 – Least successful, characterised: “There’s loads
there” (quantity) as they went
By Franz Glaw (Own work) [CC-BY-SA-3.0]
3x groups of 11 year olds, searching (on role models) for open and
closed questions; asked to justify their answers as they went

22. 26
Friends of
the Earth:
Press
Release
(Urine
presence)
FoE
Commissione
d report
(‘scientific’)
(-ve)
Science-
Literacy
website:
Refutation
(+ve)
Farmer’s
Weekly
Reprints
(+ve)
Related peer-
review
publication
(Limited risk)
Peer-review
publication
Health
danger
Reuters
Reprints
main claims
Blogger
Critiques
journal &
author
Peer-review
publication
(Limited
health risks)
Peer-review
review of
literature
(Limited risk
to health or
plants)
Peer-review
of lit
(Limited risk;
control
suggestions)
Urine
Health
Agricu
lture

23. A ¬ B, C
supports
B but…
Write a recommendation to your friend…
Text
A
Text
B
Text
C
Lots of
text A
Selector
•Text C
•Text A
•Text B
SynthesiserSatisficer

24. Get people (undergrads) talking….
E i think the disadvantages are more important then the advantages
E because i'm looking at an other website right now that talks about different sideeffects
than i mentioned before
Q I think Reuters is more reliable than that site
Q does the source where u got the animal study from say how high the doses were
Q all the sites that i found references the same site
Q I think its better that we keep looking from different sources since we have more
variety
T i guess we can do something like first looking at the potential health risks
T Animal studies have been conducted in China using high doses of red yeast rice
products No damage to the kidneys liver or other organs were demonstrated in these
studies
T What do you already have I have that it can be dangerous if it contains monacolin K
and the side effects of it
S I'm just going to write some random notes down then we can structure it once we have
enough info
S we need to rewrite everything

25. Argument mapping?
29
http://en.arguman.org/

26. Need to navigate different sources,
disagreement, and uncertainty.
1. Encourage people to consider
their evidence, and its quality,
against the particular claims or
hypotheses being asserted;
2. Engage people in identifying
others’, and constructing their
own, arguments;
3. Multiple perspectives (and the
evidence on which they are
founded) as a means to open
discussion.
(Handbook of Epistemic Cognition)
31Socrates Looking in a Mirror, Vaillant 17thC

27. 32
What’s the point of
science communication?
And how do we know if we’re there?

28. We can expose people to disagreement and uncertainty
• Exposure to conflicts can lead to learning (Keinhues,
Stadtler, Bromme, 2011)
• People might be more likely to engage with some topics
(unfamiliar?) than others (Bromme et al 2014)
Guide them through disagreements:
• Asking people to explain their evidence (and its quality)
helps; scaffolding use and critique of sources supports &
lateral search supports this (Knight & Mercer 2015/16;
Kammerer, Amann, Gerjets 2015, Barzilai, Ka’adan, 2017)
• Don’t ask for a basic summary of texts, ask for an
argument or recommendation from multiple sources
(Stadtler, Scharrer, Bromme, 2011)
• Text that specifically tackles the arguments made
elsewhere (refutations) can produce changed
perspectives (Broughton et al, 2011)
33
By Jack Dorsey CC-By
People can navigate complexity!

29. • Presentation of science reflects
how it is understood: Explain
evidentiary standards, tackle
false balance
• Policy needs to support all –
students and educators – in
preparing for this
(Sinatra & Hofer, 2016)
34
By Jack Dorsey CC-By
Implications for science communication

30. • Presentation of science reflects
how it is understood – false
balance & explaining evidentiary
standards matter
• Policy needs to support all –
students and educators – in
preparing for this
(Sinatra & Hofer, 2016)
35
By Jack Dorsey CC-By
Implications for science communication
Not just what the claims/findings are,
but what the evidence (and quality) is.
Unpack the evidence people need to
support the arguments they’re making
Present sides, but it doesn’t have to
be neutral! Showing how sides are
related, and even refute, can help.

31. What’s our impact?
Do they understand the science
→ do they understand the
evidence
Improved: Synthesis, dialogue,
argument and perspective taking
36
By Jack Dorsey CC-By

32. Thank
you
Dr Simon Knight
@sjgknight
http://sjgknight.com
Acknowledgements (among others)
Open University UK colleagues (particularly Profs Karen Littleton & Bart Rienties),
Dr Dirk Tempelaar at Maastricht University,
Dr Matt Mitsui & Dr Chirag Shah at Rutgers,
Prof Neil Mercer at Cambridge,
Dr Laura Allen at Mississippi State,
UW-Madison colleagues (particularly Prof David Williamson Shaffer),
UTS colleagues, particularly in the Connected Intelligence Centre, and PhD candidate Kristine Deroover
Want to work with me? Get in touch!
For a magazine introduction to epistemic cognition &
processing multiple sources (by me) see:
https://tinyurl.com/BPSEpistemic

33. Thank you (and questions)
Knight, S., & Littleton, K. (2017). A discursive approach to the analysis of epistemic
cognition. Learning, Culture and Social Interaction. https://doi.org/10.1016/j.lcsi.2017.11.003
Knight, S., & Littleton, K. (2017). Socialising Epistemic Cognition. Educational Research
Review. 21(1), 17-32. https://doi.org/10.1016/j.edurev.2017.02.003
Knight, S., Rienties, B., Littleton, K., Tempelaar, D. T., Mitsui, M., & Shah, C. (2017). The
Orchestration of a Collaborative Information Seeking Learning Task. Information Retrieval.
20(5), 480-505. http://dx.doi.org/10.1007/s10791-017-9304-z
Knight, S., Rienties, B., Littleton, K., Mitsui, M., Tempelaar, D. T., & Shah, C. (2017). The
relationship of (perceived) epistemic cognition to interaction with resources on the internet.
Computers in Human Behavior. 73, 507-518. http://dx.doi.org/10.1016/j.chb.2017.04.014
Knight, S., Allen, L., Littleton, K., Rienties, B., & Tempelaar, D. T. (2016). Writing Analytics
for Epistemic Features of Student Writing. In C.-K. Looi, J. L. Polman, U. Cress, & P.
Reimann (Eds.), 12th International Conference of the Learning Sciences (Vol. 2, pp. 194–
201). Singapore: International Society of the Learning Sciences.
Knight, S., Arastoopour, G., Williamson Shaffer, D., Buckingham Shum, S., & Littleton, K.
(2014). Epistemic Networks for Epistemic Commitments. In J. L. Polman, E. A. Kyza, K. D.
O’Neill, I. Tabak, W. R. Penuel, S. Jurow, … L. D’Amico (Eds.), 11th International
Conference of the Learning Sciences (Vol. 3, pp. 150–157). Boulder, CO: International
Society of the Learning Sciences.
@sjgknight http://sjgknight.com/

34. Who to believe? How epistemic cognition can inform science communication
Two patients with the same condition decide to research possible treatments. They encounter multiple sources,
from experts and others, each with different – sometimes contradictory – information. Depending on whom they
believe and how they integrate these claims, the patients may make radically different decisions. These
situations are commonplace in everyday life, from medical choices, to our voting decisions. How do we
understand these differences, and support people in making the best decisions?
Epistemic cognition provides one lens onto this problem. Epistemic cognition is the study of how people think
about the justification, source, complexity, and certainty of knowledge. When we evaluate evidence, think about
where and when it applies, and connect claims to build models, we engage our epistemic cognition.
Understanding how people navigate their own, and others’ knowledge is one of the most pressing social issues
of our time in order to develop a sustainable society. I’ll draw on research in epistemic cognition, and my own
research on how people search for and talk about evidence, to flag key implications of epistemic cognition
research for science communication.
39
Abstract

35. Brief
your research in epistemic cognition would be of interest to science communicators. If science communicators understand how
people decide that they “know” versus “think/believe/doubt” that something is true, that may support communicating science and
finding ways to help people make better decisions.
The aim of the research stream is to share research insights that improve science communication practice and evaluation, with a
particular focus on the following suggested topics (this is not an exhaustive list):
Studies investigating the effectiveness of science communication practices:
Systematic examination of strategies and impacts
Critical and/or innovative approaches to science communication evaluation and research
Applying science communication research and evaluation to practice
Research from different fields that informs or sheds a different light on science communication practices
Building a theoretical basis for science communication.
Not always just looking for an increase in scientific literacy, we care about understanding of science, conceptual change, curiosity
My notes on Audience, etc.:
Science communicators: from scientists who do that, science journos/media, ‘translators’, museum, etc. staff.
Not necessarily in science comm
Looking to bridge research-practice gap (as in lots of education work)
Assessing impact beyond survey happy sheets – behaviour & dialogue, etc.
Two 2.5 hour sessions
25 minute talk + 10 minute Q&A
40

36. Useful resources
https://physicstoday.scitation.org/do/10.1063/PT.6.3.20171208a/full/
https://theconversation.com/what-does-research-say-about-how-to-effectively-communicate-
about-science-70244
Southerland, S.A., Sinatra, G.M. & Matthews, M.R. Educational Psychology Review (2001) 13:
325. https://doi.org/10.1023/A:1011913813847 – also interesting
41

37. Coagmento Tool
Chat
Foreground searches
Share ‘snippets’
Etherpad
Tracks pageviews & copy (/ctrl+c)

38. Figure3:3: Coagmento Screenshots (from top: 3.3.1 A full screen display from a browser window;
3.3.2 The toolbar element; 3.3.3 Sidebar with Chat displayed; 3.3.4 Sidebar with Snippets displayed)

39. Constructs->indicators
Communication – particularly meta-discourse involving exploratory discussion of credentials,
sources, etc. – is key to collectively authored written outputs
(Goldman and Scardamalia [17, p.260]).

40. For example
Pair and two trios of female 11
year old English secondary
school students
Three reasons:
Theoretical: Suggestion certain
modes of dialogue (exploratory)
may be epistemic in nature
Methodological: process access
Practical: Suggestion it is
related to better seeking &
frequent in classrooms
Knight, S., Mercer, N. (2014 and 2016), in Technology,
Pedagogy and Education. By Franz Glaw (Own work) [CC-BY-SA-

41. The Analysis
1 hour of assigned tasks on ‘role models’
Closed (“How many women have won the Nobel Prize?”) & more open (“Why do some
people think Nelson Mandela is a good role model?”).
Asked to ‘justify’ choices & state sources
Analysis for epistemic & exploratory dialogue w/reference to screencast

42. Group differences
Group 1 – Quite successful, characterised: “it’s got all the important information” “it’s a good
site” (quality)
Group 2 – Quite successful, characterised: “I didn’t know that” (novelty), authority
Group 3 – Least successful, characterised: “There’s loads there” (quantity)

43. Topic – themes (red yeast rice, glyphosate) and subthemes (urine, agriculture, etc.) mentioned
Source quality – metadata and source citation (e.g., the author name ‘Gillam’, or the domain name drugs.com),
and terms relating to citation (‘written by’, ‘published in’, etc.).
Exploratory – epistemic stance/exploratory dialogue terms (E.g., ‘because’, ‘I think’, ‘therefore’)
Synthesis – writing activity terms (e.g., ‘should we quote?’, ‘I’ll edit the text’, etc.)
Chat Indicators:
Deductive content analysis

44. Why do we disagree, preliminary list
knowledge gaps (we're missing information)
probabilistic uncertainty
theoretical perspective from legitimate experts
ambiguity (e.g., is the disagreement about the thing, or the implications of the thing for policy / practice)
degree of disagreement (is it about the nuance of it - e.g., the best method to estimate x - or about the fundamentals - e.g.
whether x is even important)
expertise gaps
incompetence
bias and financial incentive
“epistemic ” factors — i .e., issues that concern the quality of the data, reliability of available research methods, and
familiar forms of scientific uncertainty — and “nonepistemic ” factors — those that involve bias or ideological distortion s of
the ideal scientific process by outside influences in bringing about persistent disagreement ?” (taken from some source or
other??)
==============
Thomm, E., Barzilai, S., & Bromme, R. (2017). Why do experts disagree? The role of conflict topics and epistemic
perspectives in conflict explanations. Learning and Instruction, 52, 15-26.
Bientzle, M., Cress, U., & Kimmerle, J. (2013). How students deal with inconsistencies in health knowledge. Medical
Education, 47(7), 683-690.
Scharrer, L., Rupieper, Y., Stadtler, M., & Bromme, R. (2017). When science becomes too easy: Science popularization
inclines laypeople to underrate their dependence on experts. Public Understanding of Science, 26(8), 1003-1018.
Bromme, R., Thomm, E., & Wolf, V. (2015). From understanding to deference: laypersons' and medical students' views on
conflicts within medicine. International Journal of Science Education, Part B, 5(1), 68-91.
Kammerer, Y., Amann, D. G., & Gerjets, P. (2015). When adults without university education search the Internet for health
information: The roles of Internet-specific epistemic beliefs and a source evaluation intervention. Computers in Human
Behavior, 48, 297-309.
Barzilai, S., Tzadok, E., & Eshet-Alkalai, Y. (2015). Sourcing while reading divergent expert accounts: Pathways from views
of knowing to written argumentation. Instructional Science, 43(6), 737-766.
Kohl, P. A., Kim, S. Y., Peng, Y., Akin, H., Koh, E. J., Howell, A., & Dunwoody, S. (2016). The influence of weight-of-evidence
strategies on audience perceptions of (un) certainty when media cover contested science. Public Understanding of Science,
49

45. NOTE!
It would be nice to eat my own dogfood here and explain the context, disagreements, and
uncertainties a bit more.
I’m aware of the irony, and in a future version will extend this talk to add that (for a longer talk).
50