1. Debunking Moral Intuition
A Hodgepodge of Multipurpose
Kludges
Based on work by:
Stephen Stich
Joshua Knobe
Daniel Kelly
1

2. Introduction
• Philosophers – and more recently cognitive
scientists – have offered many accounts of the
psychological mechanisms & processes underlying
intuitive moral judgment
• Moral philosophers have always insisted that
sometimes the outputs of those processes –
people’s “moral intuitions” – are not to be trusted
– though they disagree about when skepticism is
warranted
2

3. Introduction
• Our goal in this talk is to sketch a newly emerging
perspective on the mechanisms underlying moral
intuition …
• and to explore its implications for the hotly
debated issue of whether and when intuitions
should be relied on
3

4. Introduction
• Philosophers have typically assumed that those
mechanisms were well designed for … something
• But we now have reasons to think that many of
theses mechanisms are not well designed for
ANYTHING
4

5. Introduction
Moral Psychology is a Kludge
A hodgepodge of multipurpose
kludges!
5

6. Introduction
• Before explaining and defending this claim it will be
useful to consider some of the reasons that
philosophers – both classic & contemporary – have
offered for discounting moral intuitions
6

7. Philosophical Background
• Reflective Equilibrium
– Rawls’ “Decision Procedure for Ethics”
(1951)
– Narrow Reflective Equilibrium
• Bring intuitions about
– particular cases
– moral principles
into accord
• To do this, sometimes an intuition about a particular case
must be rejected
7

8. Philosophical Background
– Wide Reflective Equilibrium
• Bring intuitions about
– particular cases
– moral principles
into accord with the rest of our beliefs
– including beliefs about scientific matters, history,
politics – even metaphysics & semantics
• Even more of our intuitions about particular cases will
have to be rejected
8

9. Philosophical Background
– Evolutionary arguments debunking intuition
• Perhaps the most influential writer in this
tradition is Peter Singer The
Expanding
Circle
Ethics and Sociobiology
Peter Singer
FARRAR, STRAUS & GIROUX
New York
1981
Updated in “Ethics & Intuition (2005)

10. Philosophical Background
– In The Expanding Circle, Singer focuses on nepotistic
intuitions which maintain that, in various domains, we
ought to value the welfare of our kin and tribesmen more
than the welfare of people outside these circles
– The psychological processes leading to judgments of this
sort were adaptive in ancestral environments (and
perhaps they still are)
– But once we see why we have these nepotistic & tribal
intuitions, Singer suggests, we can also see that there is
intuitions
no good reason to use them in a “decision procedure for
ethics”
10

11. Philosophical Background
– In “Ethics and Intuition” (2005) Singer develops the
argument by focusing on the sort of “trolley problems”
that have loomed large in recent philosophical and
empirical studies
11

12. Philosophical Background
– Singer (following Greene) maintains that the
neuroscientific evidence suggests that
intuitions about the “footbridge” case are the
result of our emotional reaction to cases in
which harm is caused by the sort of
interaction that would have occurred in
ancestral environments
12

13. Philosophical Background
“The salient feature that explains our different intuitive judgments
concerning the two cases is that the footbridge case is the kind of
situation that was likely to arise during the eons of time over which
we were evolving; whereas the standard trolley case describes a
way of bringing about someone’s death that has only been possible
in the past century or two…. But what is the moral salience of the
fact that I have killed someone in a way that was possible a million
years ago, rather than in a way that became possible only two
hundred years ago? I would answer: none….
13

14. Philosophical Background
“At [a] more general level …this … casts serious doubt on the method
of reflective equilibrium. There is little point in constructing a
moral theory designed to match considered moral judgments that
themselves stem from our evolved responses to the situations in
which we and our ancestors lived during the period of our evolution
as social mammals, primates, and finally, human beings. We should,
with our current powers of reasoning and our rapidly changing
circumstances, be able to do better than that.” (348)”
What I am saying, in brief, is this. Advances in our understanding of
ethics … undermine some conceptions of doing ethics …. Those
conceptions of ethics tend to be too respectful of our intuitions.
Our better understanding of ethics gives us grounds for being less
respectful of them.” (349)
them.
14

15. Philosophical Background
• Assumptions that Singer and the friends of intuition
share:
share
– The psychological system underlying our moral intuitions is
well designed
– Thus there is some point to – or reason for – the intuitive
moral judgments people make when the system is working
properly
• Though Singer (unlike the friends of intuition) insists that the
function the system is designed for is of dubious moral
importance, and thus that the intuitions are not to be taken
importance
seriously
15

16. Philosophical Background
• We believe that the engine of moral intuition is not well
designed at all
• Far from being the sort of “elegant machine” celebrated
in the writings of some evolutionary psychologists, we
think that it is a kludge
– a cluster of mechanisms cobbled together rather
awkwardly from bits of mental machinery most of which
were designed for functions that have noting to do with
morality
16

17. Kelly on Disgust
• Kelly has constructed a rich,
nuanced, empirically
supported account of the
psychological mechanisms
underlying the uniquely
human disgust system and
Daniel Kelly
how that system evolved
17

18. Kelly on Disgust
• The Entanglement Thesis
– Disgust is itself a kludge – a uniquely human
emotion produced by the merger of two
distinct systems
• The Co-Optation Thesis
– After the merger, disgust was co-opted by
• the norm system
• the ethnic boundary system
which were central elements in the
emergence of human ultra-sociality
18

19. Kelly on Disgust
• Kelly assembles a vast array of evidence for these
theses, drawn from
– neuroscience
– social psychology
– cognitive psychology
– developmental psychology
– evolutionary psychology
– gene-culture co-evolution theory
• As usual, the devil is in the details
– read the work as it appears in print
19

20. Kelly on Disgust
The Entanglement Thesis
• Disgust exhibits a puzzling array of
elicitors
which evoke an equally puzzling cluster of
responses
20

21. Kelly on Disgust
The Entanglement Thesis
• Elicitors include
– Foods: dog meat, grubs, insects
Foods
21

22. Kelly on Disgust
The Entanglement Thesis
• Elicitors include
– Foods: dog meat, grubs, insects
Foods
– Substances associated with the body: feces, vomit, spit
body
– Organic decay
– People and objects associated with illness: a shirt once
illness
worn by a person with leprosy
– Sexual practices: necrophilia, incest
practices
– Some moral transgressions & transgressors: rape, torture,
child molestation
– Members of low status outgroups: untouchables, Jews
outgroups
22

23. Kelly on Disgust
Some elicitors are pan-cultural
The Entanglement Thesis
• Elicitors include
– Foods: dog meat, grubs, insects
Foods
– Substances associated with the body: feces, vomit, spit
body
– Organic decay
– People and objects associated with illness: a shirt once
illness
worn by a person with leprosy
– Sexual practices: necrophilia, incest
practices
– Some moral transgressions & transgressors: rape, torture,
child molestation
– Members of low status outgroups: untouchables, Jews
outgroups
23

24. Kelly on Disgust
Others are culturally local
(or idiosyncratic)
The Entanglement Thesis
• Elicitors include
– Foods: dog meat, grubs, insects
Foods
– Substances associated with the body: feces, vomit, spit
body
– Organic decay
– People and objects associated with illness: a shirt once
illness
worn by a person with leprosy
– Sexual practices: necrophilia, incest
practices
– Some moral transgressions & transgressors: rape, torture,
child molestation
– Members of low status outgroups: untouchables, Jews
outgroups
24

25. Kelly on Disgust
The Entanglement Thesis
• The disgust response includes
 Gape face (occasionally accompanied by retching)
 Feeling of nausea
 Sense oral incorporation
 Quick withdrawal
 A more sustained & cognitive sense of offensiveness
 A more sustained & cognitive sense of contamination
25

26. Kelly on Disgust
The Entanglement Thesis
• How are all of these connected?
connected
• The Entanglement Thesis maintains that
the human emotion of disgust is the result
of the fusion of two distinct mechanisms
– each of which has homologous counterparts in
other species
• though they have combined only in humans
26

27. Kelly on Disgust
The Entanglement Thesis
• One mechanism (“the poison avoidance mechanism”) is
mechanism
directly linked to digestion
– It evolved to regulate food intake and protect the gut
against ingested substances that are poisonous or
otherwise harmful
– It was designed to expel substances entering the gastro-
intestinal system via the mouth
– And to acquire new elicitors very quickly
• As John Garcia famously demonstrated, ingested
substances that induce gut-based distress often generate
acquired aversions
27

28. Kelly on Disgust
The Entanglement Thesis
• The other mechanism (“the parasite avoidance
mechanism”)
mechanism
– Evolved to protect against infection from pathogens and
parasites, by avoiding them
parasites
– Not specific to ingestion, but serves to guard against coming
into close physical proximity with infectious agents
– This involves avoiding not only visible pathogens and
parasites, but also places, substances and other organisms
parasites
that might be harboring them
28

29. Kelly on Disgust
These elements of the disgust response are
traceable to the poison avoidance system
The Entanglement Thesis
• The disgust response includes
 Gape face (occasionally accompanied by retching)
 Feeling of nausea
 Sense oral incorporation
 Quick withdrawal
 A more sustained & cognitive sense of offensiveness
 A more sustained & cognitive sense of contamination
29

30. and Kelly on Disgust
these are traceable to
the parasite avoidance poison system
The Entanglement Thesis
• The disgust response includes
 Gape face (occasionally accompanied by retching)
 Feeling of nausea
 Sense oral incorporation
 Quick withdrawal
 A more sustained & cognitive sense of offensiveness
 A more sustained & cognitive sense of contamination
30

31. These elicitorson Disgust
Kelly are traceable to
the poison Entanglement Thesis
The avoidance system
• Elicitors include
– Foods: dog meat, grubs, insects
Foods
– Substances associated with the body: feces, vomit, spit
body
– Organic decay
– People and objects associated with illness: a shirt once
illness
worn by a person with leprosy
– Sexual practices: necrophilia, incest
practices
– Some moral transgressions & transgressors: rape, torture,
child molestation
– Members of low status outgroups: untouchables, Jews
outgroups
31

32. and Kelly on Disgust
these are traceable to
the parasite avoidance system
The Entanglement Thesis
• Elicitors include
– Foods: dog meat, grubs, insects
Foods
– Substances associated with the body: feces, vomit, spit
body
– Organic decay
– People and objects associated with illness: a shirt once
illness
worn by a person with leprosy
– Sexual practices: necrophilia, incest
practices
– Some moral transgressions & transgressors: rape, torture,
child molestation
– Members of low status outgroups: untouchables, Jews
outgroups
32

33. Kelly on Disgust
The Entanglement Thesis
• One bit of evidence supporting the Entanglement Thesis is
that different components of that response are on
different developmental schedules
– Distaste & gape are present within the first year of life
– Contamination sensitivity emerges significantly later
• Once the full system in in place, the components of the
response are produced together – they form a
nomological cluster
– Any elicitor of disgust will reliably produce all or most of those
clustered components
33

34. Kelly on Disgust
The Entanglement Thesis
• A puzzle:
puzzle
– Why should the sight of a festering sore or a person with
leprosy evoke a gape face and a feeling of nausea?
• The solution: Disgust is a kludge!
solution kludge
• But it is kludge with features that could be readily co-
opted and put to other uses as humans began living in
larger groups and human ultrasociality emerged
34

35. Kelly on Disgust
The Co-Optation Thesis
35

36. Kelly on Disgust
The Co-Optation Thesis
• The Gape Face as a Signal
– As group size increased, there was an increasing need for a
perspicuous signal warning of dangerous foods and risk of
infectious disease
– In humans, the face and facial expressions provide a rich
source of such social information
– The gape face, which clearly has roots in the facial motions
face
that accompany retching, was co-opted as a signal, warning
signal
others not just against toxic foods, but also against the
foods
presence of parasites and contagious pathogens
36

37. Kelly on Disgust
The Co-Optation Thesis
• Co-Optation by the Norm System
– As group size increased, there was increased need for
complex social coordination
– The norm system – whose structure we considered briefly in
the 2nd Lecture – played an important role in facilitating this
co-ordination
– And the disgust system had features that made it an obvious
candidate to be co-opted by the norm system as it evolved
37

38. Kelly on Disgust
The Co-Optation Thesis
– The S&S model suggests that compliance motivation &
punitive motivation are linked to “the emotion system”
38

39. Kelly on Disgust
The Co-Optation Thesis
other emotion
Acquisition triggers
Execution Mechanism
Mechanism
norm data base complianc beliefs
r1---------- e
r2---------- motivation
r3----------
…… emotion judgment
rn---------- system
r o v aheb
gnt acl p m
Rule-related
i
m on y i t ned
i
reasoning
s e u evt a m on
f o s ne noc r e n
f i
i i
i
capacity punitive
r
f
motivation
explicit
l r i
r
t t
reasoning
Proximal
Cues in
Environment post-hoc
justification
39

40. Kelly on Disgust
The Co-Optation Thesis
– But psychological & neurological evidence indicates that
there are several separate emotion systems – the disgust
system being one of them
40

41. Kelly on Disgust
The Co-Optation Thesis
other emotion
Acquisition triggers
Execution Mechanism
Mechanism
norm data base complianc beliefs
r1---------- e
r2---------- motivation
r3---------- DISGUST
…… judgment
rn---------- other
r o v aheb
gnt acl p m
Rule-related emotions
i
m on y i t ned
i
reasoning
s e u evt a m on
f o s ne noc r e n
f i
i i
i
capacity punitive
r
f
motivation
explicit
l r i
r
t t
reasoning
Proximal
Cues in
Environment post-hoc
justification
41

42. Kelly on Disgust
The Co-Optation Thesis
– Disgust is a natural candidate to provide both compliance &
punitive motivation for norms that involve intrinsically
disgusting matters, like the disposal of corpses & bodily
wastes, and other activities that are antecedently salient to
the disgust system, like eating practices
• Compliance is motivated by making norm violating
behavior disgusting & thus aversive
• Punitive motivation is provided because the violator is
considered dirty and contaminated and is avoided or
shunned
42

43. Kelly on Disgust
The Co-Optation Thesis
other emotion
Acquisition triggers
Execution Mechanism
Mechanism
norm data base complianc beliefs
r1---------- e
r2---------- motivation
r3---------- DISGUST
…… judgment
rn---------- other
r o v a heb
gnt acl p m
Rule-related emotions
i
m on y i t ned
i
reasoning
s e u evt a m on
f o s ne noc r e n
f i
i i
i
capacity punitive
r
f
motivation
explicit
l r i
r
t t
reasoning
Proximal
Cues in
Environment post-hoc
justification
43

44. Kelly on Disgust
The Co-Optation Thesis
• The norm system is thus a kludge built with kludgy parts
– Not surprisingly, this can lead to some very quirky and
disturbing behavior
– Several recent studies have focused on the fact that the
disgust system can be triggered by many things that have
nothing to do with norms
• but even when triggered by these non-moral items, the
items
disgust system can have dramatic and persistent influence
on a person’s judgments about moral issues
44

45. Kelly on Disgust
The Co-Optation Thesis
other emotion
Acquisition triggers
Execution Mechanism
Mechanism
norm data base complianc beliefs
r1---------- e
r2---------- motivation
r3---------- DISGUST
…… judgment
rn---------- other
r o v aheb
gnt acl p m
Rule-related emotions
i
m on y i t ned
i
reasoning
s e u evt a m on
f o s ne noc r e n
f i
i i
i
capacity punitive
r
f
motivation
explicit
l r i
r
t t
reasoning
Proximal
Cues in
Environment post-hoc
justification
45

46. Kelly on Disgust
The Co-Optation Thesis
• Wheatley & Haidt have shown that when participants are
hypnotically induced to feel a brief pang of disgust when
they encounter the work “often” and then presented with
the following scenario
“Dan is a student council representative at his school. This
semester he is in charge of scheduling discussions about
academic issues. He often picks topics that appeal to both
professors and students in order to stimulate discussion.”
many judge that Dan is doing something wrong!
wrong
46

47. Kelly on Disgust
The Co-Optation Thesis
• Schnall et al. have shown participants make more severe
moral judgments when the judgments are made in a
disgusting office:
• greasy pizza boxes
• sticky chair
• a dried up smoothie
• a chewed up pen
47

48. Kelly on Disgust
The Co-Optation Thesis
• Other studies have focused on prima facie irrational
downstream consequences of the disgust system being
triggered in moral deliberation
48

49. Kelly on Disgust
The Co-Optation Thesis
Downstream
other emotion consequences
Acquisition triggers
Execution Mechanism
Mechanism
norm data base complianc beliefs
r1---------- e
r2---------- motivation
r3---------- DISGUST
…… judgment
rn---------- other
r o v a heb
gnt acl p m
Rule-related emotions
i
m on yi t nedi
reasoning
s e u evt a m on
f o s ne noc r e n
f i
i i
i
capacity punitive
r
f
motivation
explicit
l r i
r
t t
reasoning
Proximal
Cues in
Environment post-hoc
justification
49

50. Kelly on Disgust
The Co-Optation Thesis
• The Lady Macbeth Effect
– Zhong & Liljenquist have shown that recalling an unethical
deed increased the desire for products related to cleansing,
like antiseptic wipes
– And that cleaning one’s hands after describing a past
unethical deed reduced moral emotions like guilt & shame
• and also reduced the likelihood that participants would
volunteer to help a desperate graduate student!
50

51. Kelly on Disgust
The Co-Optation Thesis
• The Lady Macbeth Effect
– Schnall et al. (unpublished) compared judgments about
moral severity in two groups of participants
• One group had just used an alcohol-based cleansing gel
on their hands
• The other group had just used an ordinary, non-
cleansing hand cream
– The moral judgments of those using the cleansing gel were
significantly less severe!
51

52. Kelly on Disgust
The Co-Optation Thesis
• Ethnic Boundary Markers
– Boyd & Richerson & their students have argued that
another crucial step in the development of human ultra-
sociality was the emergence of mechanisms which allow
people to recognize members of their own tribe or “ethnie”
• This is important because in-group members share beliefs
& norms that facilitate coordination
52

53. Kelly on Disgust
The Co-Optation Thesis
– Since different cuisines & eating practices are one of the
more visible correlates of ethnie membership, and since
disgust is heavily involved in regulating food intake, disgust
was a natural candidate to be co-opted by the emerging
system of ethnic identification
– Eating practices of out-groups and other readily detectable
signs of out-group membership came to evoke disgust
– And disgust came to provided a significant part of the
motivation to avoid out-group members
53

54. Kelly on Disgust
The Co-Optation Thesis
– Though the evolutionary function of the ethnic boundary
marker system was to facilitate cooperation by keeping
groups apart, the kludgy solution to this problem has some
unfortunate consequences
– Out-group members are not simply avoided, they are also
considered offensive & contaminating
– People who embrace different norms are often felt to be
disgusting and sub-human!
54

55. Kludge Meets Kass
55

56. Kludge Meets Kass
• Leon Kass, M.D., Ph.D.
– Conservative bio-ethicist
– Chairman of the U. S. A.
President's Council on
Bioethics from 2002 to 2005
56

57. Kludge Meets Kass
• In his book, Life, Liberty & the Defense of Dignity (2002), there is
a chapter called “The Wisdom of Repugnance”
• Kass maintains that
– "in crucial cases...repugnance is the emotional expression of
deep wisdom, beyond reason's power fully to articulate it.”
wisdom
– “In this age in which everything is held to be permissible so
long as it is freely done, and in which our bodies are regarded
as mere instruments of our autonomous rational will,
repugnance may be the only voice left that speaks up to
defend the core of our humanity. Shallow are the souls that
have forgotten how to shudder."
shudder
57

58. Kludge Meets Kass
• The claims play a central role in Kass’ critique of human cloning
• Others have adopted the idea to argue against abortion,
abortion
pornography & same-sex marriage
58

59. Kludge Meets Kass
• Some philosophers, most notably
Martha Nussbaum, have challenged
Kass, arguing that disgust should be
discounted in moral & legal
deliberation because (roughly) it
reminds us of our animal origins
59

60. Kludge Meets Kass
I think Kelly’s work offers a far more
plausible &
powerful
critique
60

61. Kludge Meets Kass
• There is no reason to think there is
wisdom in repugnance
because
Disgust is a Kludge
and the psychological system that bases moral judgments on
disgust is a
Kludge twice over!
61

62. Kludge Meets Kass
 Anti-Jewish Nazi propaganda often invoked the
imagery and language of disgust, purity,
contamination & dehumanization very flagrantly
A poster advertising
the film The Eternal
Jew
Hitler described “the
Jew” as “a maggot in
a festering abscess,
hidden away inside
the clean and healthy
body of the nation”
62

63. Knobe on Norms and Intentional
Action
• My second example draws
some elegant and exciting
work by Joshua Knobe which
demonstrates the way in
which unconscious moral
judgments – judgments
which an agent may explicitly
reject – can nonetheless have
significant impact on a range
of morally relevant intuitions
63

64. Knobe on Norms and Intentional
Action
• In his new book, Kluge, Gary Marcus
argues that more recently evolved,
computationally slow and consciously
accessible mental processes – “System
2 Processes” in the currently
fashionable jargon – were grafted onto
older (System 1) psychological systems
designed for quite different purposes
• The resulting kludgy architecture
accounts for many of the quirks and
shortcomings that contemporary
cognitive science has discovered
64

65. Knobe on Norms and Intentional
Action
• I think that Knobe’s work provides an
important & disquieting illustration of this
phenomenon in the moral domain
65

66. Knobe on Norms and Intentional
Action
• The story begins with “the side effect effect” (aka the
Knobe effect) – one of best known and most surprising
finding in the emerging field of experimental philosophy
• Knobe (2003) reports an experiment in which participants
were presented with a pair of almost identical vignettes
66

67. Knobe on Norms and Intentional
Action
The vice-president of a company went to the chairman of
the board and said, ‘We are thinking of starting a new
program. It will help us increase profits, but it will also
harm [help] the environment.’
help
The chairman of the board answered, ‘I don’t care at all
about harming [helping] the environment. I just want to
helping
make as much profit as I can. Let’s start the new
program.’
They started the new program. Sure enough, the
environment was harmed [helped].
helped
67

68. Knobe on Norms and Intentional
Action
• In the harm case, participants were asked how much
blame the chairman deserved (on a scale from 0 – 6) and
whether he intentionally harmed the environment
• In the help case, participants were asked how much praise
the chairman deserved (on a scale from 0 – 6) and whether
he intentionally helped the environment
– In the harm case, 82% said the chairman brought about the
side-effect intentionally
– In the help case, 77% said the chairman did not bring about
the side-effect intentionally
68

69. Knobe on Norms and Intentional
Action
• Knobe’s initial hypothesis was that people’s moral
assessment of the side-effect plays a substantial role in
determining whether they are willing to say that the side-
effect was brought about intentionally
– A judgment that the side-effect is morally bad makes it
more likely that it will be judged to be intentional
– Though this seems incompatible with the widespread idea
that judgments of intentionality are judgments about a
purely factual matter, it does have an obvious rationale
matter
since judgments about whether an action is intentional play
a central role in determining whether an agent deserves
praise or blame
69

70. Knobe on Norms and Intentional
Action
• Subsequent research showed that, if the hypothesis is
understood as a claim about the effect of moral
judgments that people consciously make, this hypothesis
make
is mistaken
• The problem emerges clearly in study Knobe ran in
collaboration with David Pizarro & Paul Bloom
70

71. Knobe on Norms and Intentional
Action
• Liberal university students were given Knobe-style
vignettes in which an advertising executive approves an ad
campaign which has the side-effect of
encouraging interracial sex
or placing gardenias in one’s office
71

72. Knobe on Norms and Intentional
Action
• None of the participants judged that inter-
racial sex (or placing gardenias) is morally
wrong
• But participants were much more inclined
to say that the executive intentionally
encouraged interracial sex
• Explicit moral judgments cannot explain
the difference in judgments about the
intention-ality of the side-effects
72

73. Knobe on Norms and Intentional
Action
• However, (following Pizarro & Bloom) Knobe has recently
proposed that perhaps participants were making non-
conscious normative judgments that the behavior in
question violates a norm that is made salient by the
question or situation, even if it is a norm that they
explicitly reject
73

74. Knobe on Norms and Intentional
Action
• The picture Knobe now proposes looks like this:
“In reaching a conscious moral judgment, we can consider
judgment
a variety of different moral norms, weigh these norms
against each other, perhaps even determine that some of
the norms are themselves unjustified.”
74

75. Knobe on Norms and Intentional
Action
• Non-conscious moral judgments are formed through a
much simpler (system-1 style) process
– They are formed extremely quickly and therefore involve
very shallow processing
– In generating a non-conscious moral judgment, the only
norms we consider are the ones that first come to mind.
mind
We do not search for additional norms; we do not weigh
norms against each other; we do not ask whether any of the
norms might themselves be unjustified.
unjustified
75

76. Knobe on Norms and Intentional
Action
– Instead, we simply determine whether the behavior in
question violates any of the norms in the very limited set
we are considering
– If it does, we classify it as a transgression. It is this judgment
transgression
as to whether or not the behavior is a transgression that
then influences our intuitions about intentional action.
76

77. Knobe on Norms and Intentional
Action
• The theory predicts that the most salient norms evoked
by a given case will be the ones used to in making
intentionality judgments, even if subsequent reflection
leads the agent to think that there is nothing wrong with
violating the norm – or that doing so would be a very good
thing.
• Here is a vignette that Knobe has recently used to test this
idea
77

78. Knobe on Norms and Intentional
Action
In Nazi Germany, there was a law called the ‘racial identification law.’
The purpose of the law was to help identify people of certain races so
that they could be rounded up and sent to concentration camps.
Shortly after this law was passed, the CEO of a small corporation
decided to make certain organizational changes. The Vice-President of
the corporation said: “By making those changes, you’ll definitely be
increasing our profits. But you’ll also be violating [fulfilling] the
fulfilling
requirements of the racial identification law.” The CEO said: “Look, I
know that I’ll be violating [fulfilling] the requirements of the law, but I
fulfilling
don’t care one bit about that. All I care about is making as much profit
as I can. Let’s make those organizational changes!” As soon as the CEO
gave this order, the corporation began making the organizational
changes.
– 81% of subjects in the violate condition said that he violated the
requirements intentionally; 30% of subjects in the fulfill condition
said that he fulfilled the requirements intentionally.
78

79. Knobe on Norms and Intentional
Action
• Knobe’s theory is certainly not the last word on how
intentionality judgments are generated
– His work has inspired dozens of other researchers
• there are many studies I have not mentioned
• and many others are underway
79

80. Knobe on Norms and Intentional
Action
• However, IF Knobe’s theory is on the right track, then
intentionality judgments are a product of a kludgy architecture
which can be influenced by norms and judgments which the
agent
– is not aware of, and
of
– does not endorse
• This raises serious questions about the use of those judgments
in further moral deliberation, or in the law
deliberation
80

81. From Kludginess to Skepticism
• Both Kelly’s & Knobe’s work support the hypothesis that
motivates this talk
The psychological mechanism underlying moral intuition
is
A Hodgepodge of Multipurpose Kludges
81

82. From Kludginess to Skepticism
• Suppose that’s right. What should we conclude about moral
intuition?
– The answer is NOT that all moral intuition should be rejected
• nor even that intuitions that are closely tied to kludgy features
of the mind should be rejected
– For, as Shaun Nichols has argued, some of the most admirable
features of the cultural evolution of norms – including the
increased scope and acceptance of norms prohibiting physical
harm – are the products of kludgy design
82

83. From Kludginess to Skepticism
• Rather, I suggest, the right conclusion to draw is that ALL
moral intuitions should be viewed with a healthy dose of
skepticism
– The mechanisms that give rise to them may not have been
well designed to do anything
– So we should be skeptical about moral intuitions for roughly
the same reason that we should be skeptical of the output
of a kludgy piece of computer software
83

84. From Kludginess to Skepticism
• Compare and Contrast
– The friends of intuition (e.g. moral sense theorists) think the
system producing them is well designed for morally admirable
goals
• though it can sometimes misfire when conditions are unfavorable
– Previous enemies of intuition (e.g. Singer) think the system
producing them has been well designed for morally problematic
goals
– We believe that the system producing them is a kludge – much of
it has not been well designed at all!
84

85. From Kludginess to Skepticism
• But if we should be skeptical about all
intuition, how can we go about making
moral decisions?
• That’s a BIG question & a HARD one.
one
– Perhaps I’ll be able to suggest an answer …
85

86. From Kludginess to Skepticism
…the next time I come to Paris
86