Conversational AI (chat bots) is here to stay, and it's teaching us a lot about transactions, human language patterns, and the limits of computer-human interaction. But what about AI for Conversation? Can we learn from the Conversational AI research and improve how human-to-human conversation works? Where can we use pattern recognition and predictive analytics to improve how we are present as managers, coaches, analysts, family members or diplomats?

1. Midwest KM Symposium
Conversational AI and AI for Conversation:
Our Role as KM’ers?
Katrina Pugh, Columbia University and AlignConsulting July 9, 2019

2. Agenda
10:00-10:20
• What’s the difference between Conversational AI and AI for
Conversation? Short tour of conversational AI and AI for Conversation
research today
10:20-10:35
• Table discussion: What’s our responsibility as a KM community?
10:35-11:00
• Report outs and group discussion
2

3. Alchemy of conversation: from individual identity to
collective wisdom
• The World Café [conversation] process
reawakens our deep species’ memory of
two fundamental beliefs about human life.
• First, we humans want to work together on
things that matter to us. In fact, this is what
gives satisfaction and meaning to life.
• Second, as we work together, we are able to
access a greater wisdom that is found only
in the collective.
Conversational AI and AI for Conversation
Margaret Wheatley, 2005
3

4. We now have the tools and insights to make conversation
better, but it appears we have started backward
• The World Café [conversation] process
reawakens our deep species’ memory of
two fundamental beliefs about human life.
• First, we humans want to work together on
things that matter to us. In fact, this is what
gives satisfaction and meaning to life.
• Second, as we work together, we are able to
access a greater wisdom that is found only
in the collective. Margaret Wheatley, 2005
What if humans (individuals and teams)
were augmented or nudged to improve the
tone or accuracy?
What if wisdom were compiled and presented
back in aggregate to solve known problems?
4
AI for Conversation
Conversational AI and AI for Conversation

5. The father of dialogue, David Bohm, was a particle
physicist.
•Dialogue: “To turn toward
another” or “Meaning flowing
through”
•“I can hold the [Field/space] for
you, but I bring my own
experiences.”
…New, shared meaning
5Four Discussion Disciplines
Bohm, David, Donald Factor and Peter Garrett, Dialogue: A Proposal (1991)
http://infed.org/archives/e-texts/bohm_dialogue.htm
Conversational AI and AI for Conversation

6. 6
Conversational AI
Definition: Use of messaging apps, speech-based
assistants, and chatbots to automate communication
and create personalized customer or employee
experiences at scale.
Studies practices of human conversation, informs
conversation content or guardrails, then new
conversation participation involves
humans with bots
Conversational AI and AI for Conversation

7. Ingestion: Conversational AI for full spectrum of
human communication
7
Our interactions will be
way more conversational,
much more multi-modal.
Apps will be able to pick up
on our gestures, our facial
expressions, our emotions,
what is being said in our
voice.”
Gabi Zijderveld,
Chief Marketing Officer,
Affectiva
Source: IBM
Scientists have found a way to decode
brain signals into speech
It’s a step towards a system that would let
people send texts straight from their
brains.
by Antonio Regalado MIT Technology
Review, April 24, 2019
Google’s AI can now translate your
speech while keeping your voice
Researchers trained a neural network to
map audio “voiceprints” from one
language to another.
by Karen Hao MIT Technology Review
May 20, 2019
Conversational AI and AI for Conversation

8. Ingestion: Google Perspective AI learns about tone, slang
• API uses machine learning models to score the perceived impact a comment might have on a
conversation. Developers and publishers can use this score to give real-time feedback to
commenters or help moderators
8https://www.perspectiveapi.com/#/Conversational AI and AI for Conversation

9. Ingestion: Intent recognition example: Sarcasm
9
Check out Indian Institute of Technology research
Sarcasm Suite
Source: Quartz (2016) and NVDia
Humans don’t always recognize it! It
requires context (speaker, situation,
world)
Approaches:
1. Machine learning (volume, then
phrase matching / information
retrieval)
2. Rules (e.g., unexpected
juxtaposition, exaggeration)
3. Deep learning
Conversational AI and AI for Conversation

10. Ingestion and production: Open source code working
from open-source content
Example: Stanford Question and
Answer Dataset SQaD
10
Stanford Question Answering Dataset (SQuAD) is a
reading comprehension dataset, consisting of
questions posed by crowdworkers on a set of
Wikipedia articles, where the answer to every
question is a segment of text, or span, from the
corresponding reading passage, or the question
might be unanswerable.
Leader Board has coders who have taken both
the question and the text and interpreted them
effectively.
Source: SQaDConversational AI and AI for Conversation

11. Chatbots/intelligent agents range from
playback, to suggesting, to personalized
chitchat (illustrative)
General Inquiries
Handle basic customer
inquiries traditionally hidden in
FAQs.
Complex Inquiries
For requests that a Bot
cannot complete hand off to a
human agent.
Appointment Booking
Handle simple “anonymous” tasks
via chat, e.g. booking an
appointment.
Account Inquiry
Handle inquiries that
require identification of the
customer, e.g. account balance
enquiry
Make Payment
Perform actions that require a
user to be authenticated and
authorised, e.g. make payment
to predefined payee.
Self-service content
Take existing FAQs and
content,
enable staff to self-serve
knowledge via Bot.
Simple Customer Bots
Know-you Bots
Playback Bots
Advisor Bot
Assist a customer with a
multi-step process, solve a
problem
IT Helpdesk
Report an IT problem and
check the status. Bot can
provide self-help and
escalate to engineer if
required.
Future:
Dialogue
?
Source: EY researchConversational AI and AI for Conversation

12. AI for Conversation
Definition: Use of natural language processing
(NLP) to interpret conversation structure,
language, tone, and impact. AI for Conversation
will provide personalized input to
speakers/authors and teams to improve
performance at scale.
Can we improve how modern organizations
and societies use conversation to
collectively surface and solve problems?
12Conversational AI and AI for Conversation

13. The “Conversational firm” is a model for our time
13
• “Conversation” was more apt [than
‘Openness,’ as a term]…The key is to have
an open enough communication
environment so that the organization can
collectively surface the needs of the current
moment and thoughtfully approach the
next.”
• Employees at TechCo expect to have a
voice, conversation-friendly corporate
policies and related convening structures
(physical layout, wiki).Source: A look inside a “Conversational Firm” MIT Sloan Management Review Ideas Made to Matter, 2-016Conversational AI and AI for Conversation

14. AI for dialogue is improving: Context switching
14
“How do you really carry context throughout a
dialogue? This is the biggest challenge. A lot of
how you understand what I’m saying depends
on what I said maybe five sentences ago, or
fifteen sentences ago. You’re building up a
state of the conversation.”
Satinder Singh, Director of the
Artificial Intelligence Lab
at the University of Michigan.
Source: Kore.ai
Source: IBM
(Hold and resume concept (Kore.ai))
Conversational AI and AI for Conversation

15. Facebook/Stanford Research: “What Makes Good Conversation?: How
controllable attributes affect human judgments“
SIKM Leaders Conversational AI 15
Goals and
Previous
research
FB authors used six variables: Independent: 1. interestingness, 2. inquisitiveness, 3.
repetition, 4. specificity, 5. question-asking, Dependent: 6. humanness and 7.
engagement. Previous research didn’t grasp conversation: Shorter dialogue (Q&A)
affords crude evaluation. Single dependent variable
Approach
(used
PersonaCh
at content)
Used conditional training (CT) and weighted decoding (WD) to control repetition,
specificity, and response-relatedness. CT controls at the dialogue, rather than
utterance, level, and WD uses just in time vocabulary limiting, based on the score for
each possible “next word.”
Researchers trained originally on 2.5MM Twitter message-response pairs
Training on the PersonaChat dataset: 8,939 conversations and 955 personas.
Instructions: Each participant has persona. “Gets to know one another” In 6-8 turns.
Experiment: 1,000 conversations and, 100 personas for evaluation,
Findings 1. Good conversation is about balance (repetition, specificity, question-asking)
2. Interestingness, listening, and inquisitiveness are important determinants of
humanness and engagement ratings.
3. Could be “engaging” even if not “human”!
Conversational AI and AI for Conversation

16. Facebook/Stanford Research: “What Makes Good Conversation: How
controllable attributes affect human judgments” (cont’d)
16
• Source: Facebook research,
“What Makes Good
Conversation: How controllable
attributes affect human
judgments,” North American
Chapter of the Association of
Computational Linguistics, See,
Abigail, Stephen Roller, Douwe
Kiela, and Jason Weston,
7/28/19.
• https://research.fb.com/publ
ications/what-makes-a-good-
conversation-how-controllab
le-attributes-affect-human-ju
dgments/
Conversational AI and AI for Conversation

17. Columbia/Motorola Research: 4 Discussion disciplines
Skifstad and Pugh, “Beyond netiquette: Discussion discipline drives innovation” (In Smarter Innovation, Ark Group, 2014).
Discussion
discipline Description
1. Integrity Use true voice, research views,
Ask questions that propel
2. Courtesy Respect others and forum.
3. Inclusion Broaden the perspective.
Explain terms, call others in.
4. Translation Summarize/use insights
generated, and help others
with summarizations.
Benefit to
Collaboration
Primarily tonal;
builds community
and social capital.
Primarily
content-related;
drives innovation.
17
Columbia
Information and
Knowledge
Strategy Master’s
Capstone with
sponsor Motorola
Solutions: Coded
over 400 jive
posts and
regressed
productivity and
innovation on the
use of the four
discussion
disciplines.
Conversational AI and AI for Conversation

18. Use AI to nudge us into better conversation behaviors
18
https://sps.columbia.edu/news/knowledge-base/four-discussion-disciplines-drive-effective-online-collaboration
We could use AI to detect good
discussion “moves” and applaud
or "nudge."
These play out across email and
enterprise social networks,
Twitter, analyst calls.
Conversational AI and AI for Conversation
We could improve on public
conversation:
• Fake news (anti-integrity)
• Steamrollling (anti-courtesy)
• Closed-mindedness
(anti-inclusion)
• Certainty, abstraction
(anti-translation)

19. Let’s use our KM experience to change the world through
conversation!
19
What if
humans were
augmented or
nudged to
improve the
tone or
accuracy?
What if
wisdom
were
compiled
and
presented
back in
aggregate?
SIKM Leaders at KM World, 2018
Conversational AI and AI for Conversation

20. Table Discussions: What’s our insight on this topic as a
KM community?
Each table will discuss for 15 minutes and report out their insights
on Conversational AI and AI for Conversation:
- use cases
- topics
- tools
- resistance/readiness
- ethics / diversity and inclusion
- improving quality and access (e.g., through crowd sourcing)
20Conversational AI and AI for Conversation

21. Table 1: Why AI FOR CONVERSATION now?
21
Why does AI for Conversation matter
more today than ever? (E.g., with
communications tech availability,
demographics, digital commerce)
What Conversation Modalities merit our
using AI to improve them? (E.g.,
Enterprise social networks, Twitter,
voice-assisted smart devices, comments
and replies to blog posts).
Conversational AI and AI for Conversation

22. Table 2: What AI FOR CONVERSATION topics?
22
What Conversation Topics are growing in
our companies and civic spaces? Which
of those merit our using AI to improve
them?
For example projects, sales teams,
diplomacy, negotiations, civic
conversations, leadership development.
Conversational AI and AI for Conversation

23. Table 3: What CONVERSATIONAL AI topics?
23
Why does Conversational AI (Intelligent
agents/chat bots) matters now more
than ever?
For example, customer support, online
ordering and logistics, public agencies,
safety.
Conversational AI and AI for Conversation

24. Table 4: CONVERSATIONAL AI: Bots are taking over!
We need more bots!
24
What Conversational AI
Applications are growing, for
good? Which are worrisome?
What Conversation AI bots do we
want to see in the future?
Conversational AI and AI for Conversation

25. Table 5: What concerns us about ethics, diversity and
inclusion?
25
What are ethical and/or
diversity and inclusion
concerns do we need to raise
about AI for Conversation and
Conversational AI?
Conversational AI and AI for Conversation

26. How might we, as KM’ers,
contribute to discover and
improve AI for Conversation?
For example, can we
crowd-source conversation
memes (e.g., integrity, courtesy,
inclusion, translation)?
Table 6: How do we contribute to our AI FOR
CONVERSATION future?
26Conversational AI and AI for Conversation

27. Our work-life
is just a series
of
conversations.
Let's live and
interact
deliberately.
Conversational AI and AI for Conversation

28. Appendix
• Conversational AI opportunities and risks
• Links to Six leading tech companies Conversational AI
teams
• Resources 28

29. Appendix: Opportunities and risks in AI
29
Conversation Conversational AI Conversational AI: Opportunity or Risk?
Collective intelligence Crowdsourced, multi-sourced content O: Super-human
R: Bias
Voice, inflection, pitch “Data,” including these, plus history,
comparables, other factors
O: Super-human
R: manipulation
Mindfulness Context awareness O: Helps the forgetful
R: De-humanizes
Intention (personal purpose) Intention (goal) O: May help complete
R: Absolves responsibility
Container (trust) History (intelligence) O/R: Trust can be broken
“Affect” “Sentiment” O: Can reduce extremism
R: Can be manipulated
Emergence Bounded randomness O: Innovation
R: Lose the human will
Accountability, shared fate,
shared faith
Shaming, isolation O: Enforce/remind
R: Lose the higher power
Conversational AI and AI for Conversation

30. Appendix: Six leading tech companies Conversational AI teams
30
Facebook Natural
Language Processing
and Speech
Our natural language
processing and speech
researchers focus on the
interaction between
people and computers
using human languages,
both in diverse written and
spoken forms... 6,000
languages, deep
learning/neural networks,
natural language
processing, language
identification, text
normalization, word sense
disambiguation, and
machine learning, to break
down the problems, and
build and deploy robust
language translation
solutions.
Google
Language
Research Team
We advance the
state of the art in
natural language
technologies and
build systems that
learn to understand
and use language in
context.
Uses ML to detect
negative
conversation
Google's Perspective
API and this
article. The NY Times
uses it to handle
comments
Microsoft
Research Natural
Language
Processing Group
Develops efficient
algorithms to process
text and to make their
information accessible
to computer
applications. The goal
of the group is to
design and build
software that will
analyze, understand,
and generate
languages that
humans use naturally,
so that eventually
people can address
computers as though
they were addressing
another person.
Apple Machine
Learning Journal
Apple is dedicated to
advancing
state-of-the-art machine
learning technologies. As
part of this, we’re
building a team of
exceptional researchers
and engineers who can
infuse intelligence into
our devices and services
to touch the lives of
millions of users every
day. We’re people with
backgrounds in machine
learning , deep learning,
reinforcement learning,
computer vision, and
language technologies.
Amazon Alexa AI
The Alexa AI team
contributes to the magic that
is Alexa. Our goal is to make
voice interfaces ubiquitous
and as natural as speaking to
a human. .. cutting-edge
techniques, like highly
scalable deep learning, to
train our speech models.
..virtually all fields of human
language technology.
This WIRED article, which
includes interviews with
several Alexa scientists,
provides good insight into
our customer-centric
approach to research and
development, as does this
interview with Rohit Prasad,
vice president and head
scientist, Amazon Alexa.
IBM Watson
Research labs
for
Conversational
AI IBM Research is
tackling some of AI's
greatest challenges.
Our scientists and
engineers focus on
fundamental scientific
breakthroughs to help
guide the
advancement of AI.
Browse some of our
latest publications
spanning a wide range
of core AI disciplines.
Watson Assistant
is an offering for
building
conversational
interfaces into any
application, device, or
channel.
Conversational AI and AI for Conversation

31. Appendix: Resources
• Facebook research, “What Makes Good Conversation: How controllable attributes affect human judgments,”
North American Chapter of the Association of Computational Linguistics, See, Abigail, Stephen Roller, Douwe
Kiela, and Jason Weston, 7/28/19.
https://research.fb.com/publications/what-makes-a-good-conversation-how-controllable-attributes-affect-human
-judgments/
• IBM, How conversation (with context) will usher in the AI future
https://www.ibm.com/watson/advantage-reports/future-of-artificial-intelligence/ai-conversation.html
• Nanalyze, Conversational AI Examples (featuring Clinc)
https://www.nanalyze.com/2019/06/conversational-ai-enterprise-applications/
• Nanalyze, “Latest trends in AI” https://www.nanalyze.com/2019/03/latest-trends-artificial-intelligence/
• Forrester Wave on Conversational AI, June, 2019.
https://www.forrester.com/report/The+Forrester+New+Wave+Conversational+AI+For+Customer+Service+Q2+
2019/-/E-RES144416
• Columbia University, Four discussion disciplines
http://sps.columbia.edu/information-and-knowledge-strategy/news/four-discussion-disciplines-drive-effective
-online
• Stanford Stanford Question Answering Dataset SQaD
• MIT Technology Review, Scientists have found a way to decode brain signals into speech It’s a step towards a
system that would let people send texts straight from their brains. by Antonio Regalado Technology Review,
Apr 24, 2019
• MIT Technology Review, Google’s AI can now translate your speech while keeping your voice Researchers
trained a neural network to map audio “voiceprints” from one language to another. by Karen Hao Technology
Review May 20, 2019
• Quartz, “Snark Attack,” Quartz (2016)
SIKM Leaders Conversational AI 31Conversational AI and AI for Conversation