Emotion AI refers to a set of technologies -- natural language processing, voice tech, facial coding, neuroscience, and behavioral analytics -- applied to interactions to extract, convey, and induce emotion. Emotion AI is a presentation by Seth Grimes at AI for Human Language, March 5, 2020 in Tel Aviv.

1. Emotion AI
Seth Grimes
Alta Plana Corporation
@sethgrimes
March 5, 2020

3. 2008: emotionai.com

4. “If we want computers to be genuinely
intelligent, to adapt to us and to interact
naturally with us, then they will need to ability
to recognize and express emotions, and to
have what has come to be called ‘emotional
intelligence.’”
-- Rosalind Picard, Affective Computing, 1997
Affective Computing

5. 1997

6. Bloom’s Taxonomy of Educational Objectives, 1956:
Affective, Cognitive & Psychomotor

7. « Le cœur a ses raisons que la raison ne connaît
point. »
-- Blaise Pascal
“All models are wrong, but some are useful.”
-- George Box

8. Emotion models: Geneva
Emotion Wheel
https://www.unige.ch/cisa/files/6914/6720/3988/2005_Scherer_SSI.pdf
https://www.unige.ch/cisa/gew

9. Emotion models: Plutchik

10. http://sentic.net
Plutchik via SenticNet
“As a framework,
SenticNet consists of a set
of tools and techniques for
sentiment analysis
combining commonsense
reasoning, psychology,
linguistics, and machine
learning.”

11. What about Sentiment Analysis? Text sourced…
https://dl.acm.org/doi/10.1145/945645.945658,
October 2003
https://dl.acm.org/doi/10.1561/1500000011,
January 2008

12. Bing Liu, 2010

13. Sentiment -> Emotion points
• Both sentiment and emotion are subjective.
• Emotion models may be fine-grained and hierarchical.
• Emotion categories aren’t orthogonal; emotion is
compositional.
• Sentiment is situational and not simply a one-dimensional
projection of emotion.
Consider…

15. Disclaimer
I use A LOT of commercial product images in the slides
that follow. These are illustrations and not meant as
recommendations.

16. Emotion:
Many sources

17. Emotion AI
“Emotion mining is the science of detecting, analyzing, and
evaluating humans' feelings towards different events, issues,
services, or any other interest.”
Emotion synthesis enhances the ability of a machine to provide
meaningful, contextual responses, by conveying an appropriate
emotional state through words, voice, and expression.
Emotion induction aims to evoke a certain emotional response or
affective state.
Examples …

19. Emotion
in Text:
Lexicons

20. Emotion in Text
https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm

21. “Parsing Text for Emotion Terms:
Analysis & Visualization Using R”
Emotion in
Text: Parsing,
Stats
https://datascienceplus.com/parsing-text-for-emotion-terms-analysis-visualization-using-r/

22. Emotion in Text

25. Voice :: Aspiration

26. Speech https://www.phon.ucl.ac.uk/courses/spsci/iss/week9.php
“Voice cues are commonly divided into those related to: (a) fundamental
frequency (F0, a correlate of the perceived pitch), (b) vocal perturbation (short-
term variability in sound production), (c) voice quality (a correlate of the
perceived ‘timbre’), (d) intensity (a correlate of the perceived loudness), and (e)
temporal aspects of speech (e.g., speech rate), as well as various combinations of
these aspects (e.g., prosodic features).”
http://www.scholarpedia.org/article/Speech_emotion_analysis

27. Voice Bots
Human =
Good
https://voicebot.ai/2019/11/27/alexa-is-learning-to-speak-emotionally/
https://voicebot.ai/2019/11/21/correct-call-to-action-recall-by-users-is-twice-
as-high-for-human-voices-as-synthetic-for-voice-apps/

28. Voice Rendering…via Markup
https://developer.amazon.com/en-US/blogs/alexa/alexa-skills-kit/2019/11/new-alexa-emotions-and-speaking-styles

29. Voice Rendering
https://www.sciencedirect.com/science/article/abs/pii/S0167639317303187

31. Conversational AI:
Unemotional
https://reprints.forrester.com/#/assets/2/664/RES144416/reports

32. “Conversational Intelligence & Behavioral Prediction Insights from
Voice: Our Oliver engine offers ASR+ with a sophisticated layer of
emotion recognition metrics & behavioral KPIs, not only from what is
being said but also from the how it is said.”

33. https://www.ipsoft.com/2019/08/29/emotional
-intelligence-in-conversational-ai-why-amelia-
is-a-leader/
“When Amelia is working with customers or coworkers, she may sense frustration,
anger, or even sadness. In these moments, she’s able to calibrate her tone and
phraseology in order to be considerate of how her counterpart is feeling. With this
emotional intelligence, she can build closer connections and relationships between
a company and their customers and employees.
“The language Amelia uses to interact with customers can vary from system to
system and even from project to project. She can use slang, humor and even
sarcasm, or she can speak in direct professional terms.”

35. MIT Nexi Robot, 2008
SoftBank Pepper, 2015

36. Facial Coding
SRI + Toyota

37. Counterpoint

38. Counterpoint 2
Affect recognition, a subset of facial
recognition that claims to “read” our
inner emotions by interpreting the
micro-expressions on our face, has
been a particular focus of growing
concern in 2019—not only because it
can encode biases, but because it
lacks any solid scientific foundation to
ensure accurate or even valid results.

39. Call for speakers through March 31: lt-innovate.org

40. Emotion AI
Seth Grimes
Alta Plana Corporation
@sethgrimes
March 5, 2020