How to emrace ignorance and learn to love uncertainty

1. David Didau
researchED - 5th September 2015
Foxy Thinking:
how to embrace
ignorance & learn
to love
uncertainty

2. I imagine this midnight moment’s forest:
Something else is alive
Beside the clock’s loneliness
And this blank page where my fingers
move.
Through the window I see no star:
Something more near
Though deeper within darkness
Is entering the loneliness:
Cold, delicately as the dark snow,
A fox’s nose touches twig, leaf;
Two eyes serve a movement, that now
And again now, and now, and now
Sets neat prints into the snow
Between trees, and warily a lame
Shadow lags by stump and in hollow
Of a body that is bold to come
Across clearings, an eye,
A widening deepening greenness,
Brilliantly, concentratedly,
Coming about its own business
Till, with a sudden sharp hot stink of fox
It enters the dark hole of the head.
The window is starless still; the clock ticks,
The page is printed.
T h e T h o u g h t F o x
Ted Hughes

3. Even if the amount of knowledge in the world is
increasing, the gap between what we know
and what we think we know may be widening.
Nate Silver, The Signal & The Noise
The larger the island of knowledge grows, the
longer the shoreline — where knowledge meets
ignorance — extends.
Michael Smithson

4. Knowledge
IGNORANCE
There are known knowns. These are things we know
that we know. There are known unknowns. That is
to say, there are things that we know we don't
know. But there are also unknown unknowns.
Donald Rumsfeld

5. IGNORANCE
Known knowns
Explicit knowledge
Known unknowns
Educated ignorance –
“the shoreline”
Unknown knowns
Tacit, implicit
knowledge
Unknown unknowns
The vast ocean of
ignorance

6. IGNORANCE
It’s very difficult to find a
black cat in a dark room.
Especially when there is no
cat.

7. • We don’t know what we don’t know
• We prefer certainty to being right
• It stops us thinking
The problem with certainty

8. Can we trust the evidence of
our own eyes?
The Müller-Lyer Illusion

9. We can’t see when we’re
wrong

10. We can’t see when we’re
wrong

12. Shepard’s ‘Turning the tables’

13. Shepard’s ‘Turning the tables’

14. Michotte’s perception of causality

15. Michotte’s perception of causality

16. Michotte’s perception of causality

17. If it looks like a duck…

18. The Necker Cube

19. Are you a fox or a hedgehog?
The fox knows many little things, but the
hedgehog knows one big thing.
Archilochus

20. Are you a fox or a hedgehog?
For there exists a great chasm between those,
on one side, who relate everything to a single
central vision, one system, less or more
coherent or articulate, in terms of which they
understand, think and feel – a single, universal,
organising principle in terms of which alone all
that they are and say has significance – and,
on the other side, those who pursue many
ends, often unrelated and even contradictory,
connected, if at all, only in some de facto
way, for some psychological or physiological
cause, related to no moral or aesthetic
principle.
Isaiah Berlin, The Hedgehog and the Fox

21. Are you a fox or a hedgehog?
How Foxes think How Hedgehogs think
Multidisciplinary Specialised
Adaptable Stalwart
Self-critical Stubborn
Tolerant of complexity Order-seeking
Cautious Confident
Empirical Ideological
✔
✔
✔
✔
✔
✔
Foxes know more about what they
don’t know

22. Are you a fox or a hedgehog?
Foxes may have emphatic convictions
about how the world ought to be. But they
can usually separate that from their
analysis of the way the world actually is
and how it is likely to be in the near future.
Hedgehogs take a prejudicial view toward
evidence, seeing what they want to see
and not what is really there.

23. People tend to think of not knowing as
something to be wiped out or
overcome, as if ignorance were simply
the absence of knowledge. But answers
don’t merely resolve questions; they
provoke new ones.
Jamie Holmes, The Case for Teaching Ignorance

24. After 30 years of doing such work, I have
concluded that classroom teaching… is
perhaps the most complex, most
challenging, and most demanding, subtle,
nuanced, and frightening activity that our
species has ever invented…The only time
a physician could possibly encounter a
situation of comparable complexity would
be in the emergency room of a hospital
during or after a natural disaster.
Lee Shulman, The Wisdom of Practice

25. How can we identify good
teachers?
• Observation
• Student outcomes
• Student surveys

26. Variation in the reliability of teacher ratings with
number of lessons observed and number of raters (Hill et al., 2011)

27. Using student outcomes
Percentages of teachers placed in different quintiles of effectiveness with two equally plausible
value-added models (Goldhaber et al., 2013)
9% of the teachers who are rated as the least effective with the
traditional random-effects model are actually rated as the most
effective with the fixed effects model.

28. The problem with ‘weighing’
teachers
• If a scale’s accuracy
was ±50 pounds, we
would not be able to
say much about the
weight of an
individual.
• In a group of 50 men
and 50 women, the
men are heavier on
average.

29. Problems with student outcomes
• Judgements depend on our
assumptions
• Results vary considerably from year to
year and from context to context.

30. Using student surveys
Clusters and indicative example statements from the Tripod student survey
(Bill and Melinda Gates Foundation, 2012a)

31. Using student surveys
• Students of teachers with the top 25%
of Tripod score, make 4.6 months more
progress in mathematics than those in
the classrooms taught by teachers with
Tripod scores in the lowest quartile.
• On average, the most effective 25% of
teachers are around 130% more
effective than the least effective 25%

32. Problems with student surveys
• The Doctor Fox Effect
• Perverse incentives

33. We can’t reliably spot good
teachers
• Aggregating observation, outcomes &
student surveys :
• We can just about tell the difference between
the least effective and the most effective
teachers

34. How do you make teachers
improve?
• Can you become a better teacher?
Or, can the teachers in your school
improve?
• Do you know exactly how to make this
happen?
• Does anyone else?
• How does your school support you?
Or, how do you support others?
• Accountability vs. trust

35. A false dichotomy?
Synthesis
Teachers should be
allowed to explain how
their practice has
resulted in pupil
progress.
Thesis
Teachers should
experiment &
develop the best
ways of giving
feedback in their
context.
Antithesis
Ofsted will
evaluate
progress by
checking if there
is pupil response
in books.

36. We all want to be good, but…
3 conditions which encourage people
to want to be right rather than look right:
• the knowledge that we will be
accountable to an audience
• the audience’s views must be
unknown
• the belief that the audience is well-
informed and interested in accuracy.
Lerner & Tetlock, 2003

37. How we get accountability
wrong
1. Teachers are told to use ‘two stars
and wish’ to mark their books.
2. There is/isn’t evidence of ‘two stars
and wish’ in this teacher’s books,
therefore I am/am not happy.

38. Better accountability
1. Teachers are told to mark students’
books in the way they believed would
make the most impact.
2. There is/isn’t evidence of marking
therefore I am/am not happy.
3. There is/isn’t evidence that the marking
has had an impact therefore I need/do
not need to ask the teacher some
follow-up questions.

39. Intelligent accountability
1. Teachers are expected to cover the
curriculum and ensure to the best of
their ability that students are
prepared for some kind of
assessment.
2. There is/isn’t evidence of marking
therefore I need/do not need to ask
the teacher some follow-up questions.

40. Key points
• Acknowledging uncertainty leads to
better decision making
• Teachers improve when they feel
trusted, supported and held
accountable
• Always remember, you might be
wrong.

41. Further reading
• Stuart Firestein, Ignorance: How it
Drives Science
• Jonathan Haidt, The Righteous Mind
• John Kay, Obliquity
• Robert Proctor & Londa Schiebinger,
Agnotology: The making & unmaking
of ignorance
• Nate Silver, The Signal & The Noise

42. @LearningSpy
learningspy.co.uk
ddidau@gmail.com