Since we first started using computers, we have dreamt of interacting with them in a more natural, human-like way using spoken language. Advances in machine learning and artificial intelligence have spiked an interest in natural language processing, and building conversational interfaces no longer is a dream, but a reality every developer can use in their own applications. The problem of recognising spoken input has been largely solved, and the next challenge is how to build compelling user experiences that are modelled after natural human conversation. In this talk, I will give an introduction to the Google Assistant and how to use Actions on Google to build conversational interfaces for devices with and without screens. We will take a look at how API.AI makes implementing natural language understanding feasible even for non-linguists, and discuss how you can integrate agents with existing backends. In conversation, it's all about being natural, so we will also look at how to create agents that feel like real characters.

2. To organize the world's information and make it universally accessible and useful.

4. Google Home/Mobile device
The Google Assistant —
Actions on Google —

5. Assistant App

7. Well, it’s kind of cold outside, so I'd like
something to warm me up, like a hot
soup, and I want it fast.
I have some chicken, and also canned
tomatoes.

9. Assistant app
{ conversation
API request }
{ conversation
API response }
user input
app
response

11. Well, it’s kind of cold outside, so I'd like
something to warm me up, like a hot
soup, and I want it fast.
I have some chicken, and also canned
tomatoes.

12. “
Intent Matching —
Entity Extraction —
@

29. const App = require('actions-on-google').ApiAiApp;
exports.yourApp = (request, response) => {
const app = new App({request, response});
console.log('Request headers: ' + JSON.stringify(request.headers));
console.log('Request body: ' + JSON.stringify(request.body));
// Fulfill action business logic
function responseHandler (app) {
// Complete your fulfillment logic and send a response
app.ask('Hello, World!');
}
const actionMap = new Map();
actionMap.set('<API.AI_action_name>', responseHandler);
app.handleRequest(actionMap);
};

30. const app = new ApiAiApp({request: request, response: response});
const WELCOME_INTENT = 'input.welcome'; // the action name from the API.AI intent
const NUMBER_INTENT = 'input.number'; // the action name from the API.AI intent
const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent
function welcomeIntent (app) {
app.ask('Welcome to action snippets! Say a number.');
}
function numberIntent (app) {
let number = app.getArgument(NUMBER_ARGUMENT);
app.tell('You said ' + number);
}
let actionMap = new Map();
actionMap.set(WELCOME_INTENT, welcomeIntent);
actionMap.set(NUMBER_INTENT, numberIntent);
app.handleRequest(actionMap);

32. if (app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)) {
app.ask(app.buildRichResponse()
.addSimpleResponse(`Here's a fact for you. ${fact} Which one ` +
`do you want to hear about next, Google's history or headquarters?`)
.addBasicCard(
app.buildBasicCard('Google is an amazing company.')
.setImage(GOOGLE_LOGO_SRC))
.addSuggestions(['History', 'Headquarters']));
} else {
app.ask(`Here's a fact for you. ${fact} Which one ` +
`do you want to hear about next, Google's history or headquarters?`);
}

35. g.co/actionswebsim

40. Hey! This is ___
Welcome to ___
Ready to play ____
Hi! ___ here.
Hello. I’m ___
Greetings, human.
Welcome back to ___
Hey again. ___
Let’s play ___
Here’s your ___
Brought to you by ___
Hi there, ___
Let’s get started.
Ready for your ___
___, here to…
Live from ___
This is ___
What’s up, ___
and more...

41. <speak> <!-- Must be at the start of the string -->
<say-as interpret-as="characters">SSML</say-as>
<break time="3s"/>.
<audio src="https://example.com/file.mp3"></audio>
<say-as interpret-as="cardinal">10</say-as>.
<say-as interpret-as="ordinal">10</say-as>
<say-as interpret-as="characters">10</say-as>.
<sub alias="World Wide Web">WWW</sub>.
<p><s>This is one.</s><s>This is two.</s></p>
</speak> <!-- Must be at the end of the string -->
“S S M L”
[3 second pause]
[audio file plays]
“Ten”
“Tenth”
“One Oh”
World Wide Web
[two sentences]

42. function welcome (app) {
return isPreviousUser(app.getUser().userId).then((userHasVisited) => {
if (userHasVisited) {
app.ask(`Welcome to Number Genie!...`, NO_INPUT_PROMPTS);
} else {
app.ask(`Hey you're back...`, NO_INPUT_PROMPTS);
}
});
}

45. function simpleResponse (app) {
app.ask({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number, like 42. What do you have in mind?',
displayText: 'Howdy! I can tell you fun facts about ' +
'almost any number. What do you have in mind?'
});
}

46. function suggestionChips (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number like 0, 42, or 100. What number do you have ' +
'in mind?',
displayText: 'Howdy! I can tell you fun facts about almost any ' +
'number. What number do you have in mind?'
})
.addSuggestions(['0', '42', '100', 'Never mind'])
);
}

47. function basicCard (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse('Math and prime numbers it is!')
.addBasicCard(
app.buildBasicCard(`42 is an even composite number. It ` +
`is composed of three distinct prime numbers multiplied together. It ` +
`has a total of eight divisors. 42 is an abundant number, because the ` +
`sum of its proper divisors 54 is greater than itself. To count from ` +
`1 to 42 would take you about twenty-one…`)
.setTitle('Math & prime numbers')
.addButton('Read more')
.setImage('https://example.google.com/42.png', 'Image alternate text')
)
);
}

50. let permission = app.SupportedPermissions.DEVICE_COARSE_LOCATION;
app.askForPermission('To find bookstores near you', permission);

55. Ok Google, ask Personal Chef for a hot soup recipe
Trigger Phrase App Name
Developer Specified
Action
Preposition
Action Phrase
Developer Specified
Ok Google, let’s talk to Personal Chef
Trigger Phrase App Name
Developer Specified

59. Conversation Design

61. GUI :: VUI What’s the difference?

62. Top 3 Design Tips

63. Create a persona.1

64. Persona is conveyed through:
Tone
Word and phrase choices
Functional design
Style
Technique
Voice
...and based on:
Your user population and their needs
The imagery & qualities associated with your brand

65. Think outside the box. Literally.2

66. Confidential and proprietary
code code code code code
code code code code
code code “ dialog string” code
code code code
code code code code code
code “dialog string” code code
code code code code code code code
code code code
code code
code
code code
code
code
code code code
code “dialog string” code code code
code code
code code code
code code code code code code code code
code code code code code code code code
code
STRUCTURE + CODE

67. Confidential and proprietary
code code code code code
code code code code
code code “ dialog string” code
code code code
code code code code code
code “dialog string” code code
code code code code code code code
code code code
code code
code
code code
code
code
code code code
code “dialog string” code code code
code code
code code code
code code code code code code code code
code code code code code code code code
code
STRUCTURE + CODE

68. Confidential and proprietary
DIALOG + STRUCTURE

69. In conversations, there are no “errors”.3

70. real life
user
conditions{

71. “I don’t understand you”

72. When a so-called “error” occurs in a
conversation, it should be treated simply as
a new turn in the dialog, only with different
conditions.

73. Users have intent

76. developers.google.com/actions

77. goo.gl/PMdr7N