When you want to talk to someone, where do you turn? Skype? Slack or HipChat? Maybe even an old-fashioned telephone? As great (or not) as these are, they all fail in one important way: Context. As developers, why don’t we enable our users to communicate where they are doing everything else, right inside the browser or mobile app? The technology to make contextual communications is evolving quickly with exciting technologies like WebRTC, speech recognition and natural language processing. This talk is about how to apply those building blocks and bring contextual communication to your apps. Presented at RailsConf 2015 in Atlanta, GA

1. Putting Real-Time
Voice, Video and Text
into Rails
Now Hear This!

2. CAN YOU SPEAK MAGIC?
2
Ben Klang
A city well known for our fine airport, and that sacred elixir we call Coca Cola

3. CAN YOU SPEAK MAGIC?
2
Ben Klang
A city well known for our fine airport, and that sacred elixir we call Coca Cola

4. CAN YOU SPEAK MAGIC?
2
Ben Klang
A city well known for our fine airport, and that sacred elixir we call Coca Cola

5. CAN YOU SPEAK MAGIC?
2
Ben Klang
A city well known for our fine airport, and that sacred elixir we call Coca Cola

6. CAN YOU SPEAK MAGIC?
3
The web today is a lot like outer space

7. CAN YOU SPEAK MAGIC?
4
On the web, no one can hear you scream

8. CAN YOU SPEAK MAGIC?
5
The scenario: you’re busy at work, working from home in your corp web application, and you decide you need to speak to a co-worker to answer a question.
Lose context: have to explain what you are looking at and limited ability to invite additional participants
Record of communication not made a part of the app

9. CAN YOU SPEAK MAGIC?
5
The scenario: you’re busy at work, working from home in your corp web application, and you decide you need to speak to a co-worker to answer a question.
Lose context: have to explain what you are looking at and limited ability to invite additional participants
Record of communication not made a part of the app

10. CAN YOU SPEAK MAGIC?
6
Web RTC

11. CAN YOU SPEAK MAGIC?
6
Web RTC

12. CAN YOU SPEAK MAGIC?
6
Web RTC

13. CAN YOU SPEAK MAGIC?
6
Web RTC

14. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
7

15. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•Use Camera & Microphone without plugins
7

16. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•Use Camera & Microphone without plugins
•Setup peer-to-peer connectivity, 2+ people
7

17. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•Use Camera & Microphone without plugins
•Setup peer-to-peer connectivity, 2+ people
•Common codecs for transmitting high-
definition media in real-time
7

18. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
8

19. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•JavaScript Browser API
for real-time communication
(voice & video)
8

20. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•JavaScript Browser API
for real-time communication
(voice & video)
•Standards for interoperability
8

21. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•JavaScript Browser API
for real-time communication
(voice & video)
•Standards for interoperability
•Opus, G.711, H.264/VP8
8

22. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•JavaScript Browser API
for real-time communication
(voice & video)
•Standards for interoperability
•Opus, G.711, H.264/VP8
•SDP
8

23. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•JavaScript Browser API
for real-time communication
(voice & video)
•Standards for interoperability
•Opus, G.711, H.264/VP8
•SDP
•ICE/STUN/TURN
8

24. CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC?
•JavaScript Browser API
for real-time communication
(voice & video)
•Standards for interoperability
•Opus, G.711, H.264/VP8
•SDP
•ICE/STUN/TURN
•DTLS-SRTP
8

25. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC TO ME?
9
Telephones in Web Browsers!

26. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
WHAT IS WEBRTC TO ME?
9
Telephones in Web Browsers!
X
Communication

27. CAN YOU SPEAK MAGIC?
10

28. CAN YOU SPEAK MAGIC?
11
Communication
Topology

29. CAN YOU SPEAK MAGIC?
12
Alice
Bob
The Trapezoid
This gives phone companies a lot of control:
Your identity (phone number)
Features available had to be common across carriers (stifled innovation)
Provides a point of intercept (NSA)
Communications are completely divorced from the function they serve

30. CAN YOU SPEAK MAGIC?
13
Alice Bob
The Triangle
Identity is better
Feature set is better
Still a common point of intercept
Still not very contextual

31. CAN YOU SPEAK MAGIC?
14
The WebRTC TriangleAlice Bob
Signaling
Media

32. CAN YOU SPEAK MAGIC?
14
The WebRTC TriangleAlice Bob
Signaling
Media

33. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

34. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
Alice
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

35. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

36. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

37. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

38. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

39. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

40. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

41. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

42. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
SRTP
SRTP
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

43. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
SRTP
SRTP
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

44. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
http://
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
SRTP
SRTP
X
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

45. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
SRTP
SRTP
X
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

46. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
15
H
aveBob
callm
ehere!
SDP:
v=0o=alice20518
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio54609
RTP/SAVPF
109
Bob
is
at:
SDP:
v=0o=bob
19915
0
IN
IP4
0.0.0.0
s=-t=0
0m
=audio
61001
RTP/SAV
PF
109
Alice Bob
SRTP
SRTP
X
When people say that WebRTC does not mandate signaling, this is what they mean: the session setup is up to the application

47. CAN YOU SPEAK MAGIC?
16
Designing
Communications Apps
That’s enough about the plumbing

48. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
17
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

49. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
17
A Modern Voice Application
should be…
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

50. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
•Adaptive
17
A Modern Voice Application
should be…
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

51. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
•Adaptive
•Fluid
17
A Modern Voice Application
should be…
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

52. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
•Adaptive
•Fluid
•Contextual
17
A Modern Voice Application
should be…
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

53. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
•Adaptive
•Fluid
•Contextual
•Trustworthy
17
A Modern Voice Application
should be…
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

54. CAN YOU SPEAK MAGIC?
THE FIVE TENETS
•Adaptive
•Fluid
•Contextual
•Trustworthy
•Reference-able
17
A Modern Voice Application
should be…
Adaptive: Can use multiple channels; Still useful if any of the channels are missing
Fluid: Can add or remove channels on demand
Reference-able: Conversations can be stored and shared later via URL, with appropriate security
Contextual: Communicating isn’t what you’re doing; it’s what you’re doing while you’re doing something else.

55. CAN YOU SPEAK MAGIC?
18
Adaptive

56. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

57. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

58. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

59. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

60. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

61. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

62. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

63. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

64. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

65. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

66. CAN YOU SPEAK MAGIC?
BEING ADAPTIVE
19
/
Alice joins via browser: gets full experience, chat, wideband voice, hi-def video, visuals
Bob joins via tablet: gets same as above thanks to mobile web, but can’t send video because his browser doesn’t support WebRTC yet??
Charlie joins via smartphone: gets same as above, except no video at all because he has reduced bandwidth or a low battery
Douglas joins via landline, because he’s backward and he really like his desk phone - maybe even get audio hints of text chat?
Edward also joins via browser, but has a sleeping baby, so he only listens to the audio and talks via chat

67. CAN YOU SPEAK MAGIC?
20
Fluid

68. CAN YOU SPEAK MAGIC?
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

69. CAN YOU SPEAK MAGIC?
•Start with Chat
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

70. CAN YOU SPEAK MAGIC?
•Start with Chat
•Add audio
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

71. CAN YOU SPEAK MAGIC?
•Start with Chat
•Add audio
•Invite more participants
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

72. CAN YOU SPEAK MAGIC?
•Start with Chat
•Add audio
•Invite more participants
•Upgrade to video
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

73. CAN YOU SPEAK MAGIC?
•Start with Chat
•Add audio
•Invite more participants
•Upgrade to video
•Downgrade back to Chat
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

74. CAN YOU SPEAK MAGIC?
•Start with Chat
•Add audio
•Invite more participants
•Upgrade to video
•Downgrade back to Chat
•Switch devices and continue
BEING FLUID
21
Visual: text -> audio -> + people -> video -> text

75. CAN YOU SPEAK MAGIC?
22
Contextual

76. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
“Communicating isn’t going
to be what you’re doing -
it’s what you’ll be doing
while you’re doing
something else”
- Geoff Hollingworth
Ericsson Head of AT&T Foundry
23

77. CAN YOU SPEAK MAGIC?
24
How many sales reps
are still in homes?
How much have we sold
so far this month?
How many callers
are in the queue
right now?
Add my manager to this call
When is the next
available appointment?
BEING CONTEXTUAL

78. CAN YOU SPEAK MAGIC?
25

79. CAN YOU SPEAK MAGIC?
26

80. CAN YOU SPEAK MAGIC?
27
Trustworthy

81. CAN YOU SPEAK MAGIC?
TRUSTWORTHY
28

82. CAN YOU SPEAK MAGIC?
TRUSTWORTHY
•Don't surprise the user!
28

83. CAN YOU SPEAK MAGIC?
TRUSTWORTHY
•Don't surprise the user!
•Don't share the data with unexpected
recipients
28

84. CAN YOU SPEAK MAGIC?
TRUSTWORTHY
•Don't surprise the user!
•Don't share the data with unexpected
recipients
•Help users make smart choices where
required (with sane defaults)
28

85. CAN YOU SPEAK MAGIC?
TRUSTWORTHY
•Don't surprise the user!
•Don't share the data with unexpected
recipients
•Help users make smart choices where
required (with sane defaults)
•If identity is asserted, it should be user-
centric and strong
28

86. CAN YOU SPEAK MAGIC?
29
Reference-able

87. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

88. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

89. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
•Every conversation has a URL that:
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

90. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
•Every conversation has a URL that:
•Is permanent & unique
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

91. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
•Every conversation has a URL that:
•Is permanent & unique
•Represents the latest state of the communication or
request
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

92. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
•Every conversation has a URL that:
•Is permanent & unique
•Represents the latest state of the communication or
request
•May contain content alternatives: recording +
transcription + annotation
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

93. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
•Every conversation has a URL that:
•Is permanent & unique
•Represents the latest state of the communication or
request
•May contain content alternatives: recording +
transcription + annotation
•Is searchable & downloadable
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

94. CAN YOU SPEAK MAGIC?
REFERENCEABILITY
•Make Interactions Shareable
•Every conversation has a URL that:
•Is permanent & unique
•Represents the latest state of the communication or
request
•May contain content alternatives: recording +
transcription + annotation
•Is searchable & downloadable
•Can be shared (respecting privacy limits)
30
All participants are invited to the chat via unique URL
This URL always contains the latest state of the chat, whether in-progress, idle, or archived
May provide enhanced features, such as downloadable copies of the text, audio or video, a summary of links or media shared, or even transcriptions of the conversation
Is searchable!

95. CAN YOU SPEAK MAGIC?
31
What Should We Build?

96. CAN YOU SPEAK MAGIC?
31
What Should We Build?
3 Ideas

97. CAN YOU SPEAK MAGIC?
1. Live, Anonymous
Matchmaking
32

98. CAN YOU SPEAK MAGIC?
33
http://funnydate.com/have-a-chat
FunnyDate
FunnyDate
SHErlock
22
Female
Male
Oakdale
Books, Moustaches, Puzzles
Captain Beard
23
Male
Female
Springfield
Books, Moustaches, Puzzles
USERNAME
AGE
GENDER
SEEKING
CITY
MATCHING
INTERESTS

99. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
34
FunnyDate

100. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
34
FunnyDate

101. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
•Strict anonymity
34
FunnyDate

102. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
•Strict anonymity
•No need to exchange
phone numbers
34
FunnyDate

103. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
•Strict anonymity
•No need to exchange
phone numbers
•Low friction interface
enables users to connect in real-time
34
FunnyDate

104. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
•Strict anonymity
•No need to exchange
phone numbers
•Low friction interface
enables users to connect in real-time
•Silly video tricks used to break the ice
34
FunnyDate

105. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
•Strict anonymity
•No need to exchange
phone numbers
•Low friction interface
enables users to connect in real-time
•Silly video tricks used to break the ice
•...or even add more anonymity
34
FunnyDate

106. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
LIVE, ANONYMOUS DATING
•Safe Introductions
•Strict anonymity
•No need to exchange
phone numbers
•Low friction interface
enables users to connect in real-time
•Silly video tricks used to break the ice
•...or even add more anonymity
•Upsell: Live language translation
34
FunnyDate

107. CAN YOU SPEAK MAGIC?
2. Incident Response App
35

108. CAN YOU SPEAK MAGIC?
36

109. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
37

110. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
37

111. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
37

112. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
37

113. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
•Inherit from existing organization groups
37

114. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
•Inherit from existing organization groups
•Allow ad-hoc participants (guests)
37

115. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
•Inherit from existing organization groups
•Allow ad-hoc participants (guests)
•Federate with external services
37

116. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
•Inherit from existing organization groups
•Allow ad-hoc participants (guests)
•Federate with external services
•Incident recording/logging
37

117. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
•Inherit from existing organization groups
•Allow ad-hoc participants (guests)
•Federate with external services
•Incident recording/logging
•“Lessons learned” and process improvement
37

118. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
INCIDENT RESPONSE
•Timely, Contextual Information
•Adapt for mobile vs. desktop
•Group-based communication
•Inherit from existing organization groups
•Allow ad-hoc participants (guests)
•Federate with external services
•Incident recording/logging
•“Lessons learned” and process improvement
•Links from/to issue tracking systems
37

119. CAN YOU SPEAK MAGIC?
3. Medical Records
Patient Services
38

120. CAN YOU SPEAK MAGIC?
39

121. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
40

122. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
40

123. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
40

124. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
40

125. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
•Verify with voice biometrics
40

126. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
•Verify with voice biometrics
•Cross-check against caller location
40

127. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
•Verify with voice biometrics
•Cross-check against caller location
•Automate Medical Claims
40

128. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
•Verify with voice biometrics
•Cross-check against caller location
•Automate Medical Claims
•Call recording/transcription
40

129. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
•Verify with voice biometrics
•Cross-check against caller location
•Automate Medical Claims
•Call recording/transcription
•Medical advice given to patient automatically
added to patient file
40

130. CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?
MEDICAL ADVICE/HISTORY
•Secure Caller Authentication
•Reuse primary auth via website
•Verify with voice biometrics
•Cross-check against caller location
•Automate Medical Claims
•Call recording/transcription
•Medical advice given to patient automatically
added to patient file
•Easy Auditing/Service Quality Assurance
40

131. CAN YOU SPEAK MAGIC?
41
Demo Time

132. CAN YOU SPEAK MAGIC?
RESOURCES
•WebRTC
•http://webrtc.github.io/samples/
•http://www.webrtc.org/
•http://www.webrtcchallenge.com/
•Voice in Ruby
•http://www.adhearsion.com/
•https://github.com/benlangfeld/ruby_speech
•Ben Klang, Mojo Lingo
•@bklang
•bklang@mojolingo.com
42

133. CAN YOU SPEAK MAGIC?
RESOURCES
•WebRTC
•http://webrtc.github.io/samples/
•http://www.webrtc.org/
•http://www.webrtcchallenge.com/
•Voice in Ruby
•http://www.adhearsion.com/
•https://github.com/benlangfeld/ruby_speech
•Ben Klang, Mojo Lingo
•@bklang
•bklang@mojolingo.com
42

134. CAN YOU SPEAK MAGIC?
RESOURCES
•WebRTC
•http://webrtc.github.io/samples/
•http://www.webrtc.org/
•http://www.webrtcchallenge.com/
•Voice in Ruby
•http://www.adhearsion.com/
•https://github.com/benlangfeld/ruby_speech
•Ben Klang, Mojo Lingo
•@bklang
•bklang@mojolingo.com
42