This document provides tips for designing effective survey questions and strategies for distributing surveys. It discusses keeping surveys short, starting broad and getting more specific, only asking necessary questions, avoiding hypothetical questions, and testing surveys internally. It also offers ways to get surveys in front of people like email, in-app opt-ins, and pop-ups. Finally, it discusses segmenting survey results by platform, time of response, and cross-referencing with analytics data.

1. Chuck Liu
How to Get Survey Answers
That Actually Help Your
Business
August 7th, 2014

2. Chuck Liu
UX Research Lead
KISSmetrics
!
@chuckjliu
cliu@kissmetrics.com
Thue L Madsen
Marketing Associate
KISSmetrics
!
@thuelmadsen
tmadsen@kissmetrics.com

3. @thuelmadsen #KISSwebinar
Join the conversation on Twi er

4. @chuckjliu #KISSwebinar
Join the conversation on Twi er

7. What do you want to learn?

8. 1 5 Tips for Designing Survey Questions
Email
In-App/Site Opt-In
In-App/Site Pop-Up
2 3 Ways to Get Surveys in Front of People
3 3 Strategies for Segmenting Survey Results
Table of Contents
4 Bonus: Our Favorite Tools and Workflows

9. WATCH WEBINAR RECORDING NOW

10. 5 Tips for Designing
Survey Questions
1

12. Lean Analytics Cycle
AVINASH KAUSHIK HTTP://KISS.LY/LEANAC

13. Lean is all about
identifying and
quantifying risk.

16. Don’t just ask
questions.

18. Know what
decisions you
can make from
these surveys.

19. Lean Analytics Cycle
AVINASH KAUSHIK HTTP://KISS.LY/LEANAC
This is where
we survey

20. 1. Keep surveys
short
!
Under 5 minutes is ideal

22. Completion
Rates

23. Survey Monkey: Does
Adding One More
Question Impact
Survey Completion
Rate?

25. Industry
Benchmarks

26. 40% is awesome
!
30% is great
!
20% is good
!
~15% is expected
!

27. What We See In
Practice

28. Longer Surveys Decrease Completion Rates
We’ve tried them all. Here’s how our survey completion rate looks based on continuous surveying.
CompletionRateofSurveys
0
10
20
30
40
1 2 3 4 5 6 7 8 9 10
# of Questions

29. A 5 Question Screener That I Did Recently

30. 14% CTR from Nudge to Screener
!
202 Responses in 7 days
!
202 Opt-Ins for further studies
!
!

31. By the way…
!
Satisfaction surveys see 10% or
lower response rates

32. 2. Start broad
then get detailed
!
Satisfaction -> specifics

34. Broad questions

35. How satisfied are
you?

36. What is your
primary
motivation?

37. What best
describes why
you came here
today?

38. Specific
questions

39. What is your
experience with
using X?

40. How satisfied are
you with X/Y/Z?

41. 3. Ask what you
need to know
!
Avoid what’s nice to know

42. What do you want to learn?

43. What are you going to DO
with this survey data?

44. It’s easy to let additional questions creep up. Resist the urge!

45. 4. Avoid
hypothetical
questions

46. Would you buy X if
we offered it?

47. Would you use X if
we offered it?

48. Instead, ask people about their CURRENT experience, frustrations,
or workarounds to their problem.

49. 5. Test survey
internally

50. Startups have smaller customer counts. Surveying multiple times
gets interpreted as spam.

51. Dry runs with your team are a great way to check if anything is
confusing.

52. 3 Ways to Get
Surveys In Front of
People
2

53. 1. Email
!
It works if you can write good
headlines.

54. Apple ’s satisfaction survey uses plain text for simplicity

55. General Assembly’s HTML emails look too automated

56. Plain text works
best.
!
Looks like a real
person wrote it.

57. 2. In-App Opt-In
!
Ask people if they’d like to
participate

58. Google Has Dedicated Usability Opt Ins

59. Twi er Integrates In-App Opt-In Into Their Apps

60. Target Opt-Ins Based On People’s Behavior To Increase
Chances of Response

61. LOG IN WITH GOOGLE
Start Your Free KISSmetrics Trial

62. 2. In-App Pop-up
!
Interrupt people during their
experience (these are the ones
that we all hate.)

63. Qualaroo Pop Up for GQ Asks about Subscription Friction

64. Qualaroo Pop Up for Logitech Asks for Use Cases

65. 3 Strategies for
Segmenting Survey
Results
3

66. Demographic segmentation is the norm, but only if it
actually helps.

67. 1. Platform
!
PC-Laptop / Phone / Tablet
!
iOS / Android / Windows / Blackberry
!
App Versions
!

69. Segmenting by Device Type

70. Another Example of Segmenting by Device Type

71. 2. Time of Survey
Response
!

72. Segmenting by Time Survey Was Completed

73. Zendesk Uses Time of Survey To Send Surveys At Times
That Have Higher Response Rates

74. Zendesk Figured Out “Not Work Hours” Worked Best

75. 3. Cross-Reference
with Analytics
!

76. Cross Reference Analytics Data with Survey Data to Get
the Whole Picture

77. Features used / products bought
!
Frequency of visits
!
Tenure - how long has this
customer been around?
!
Acquisition source - how did
this person discover my
business?
!

78. Our Favorite Tools
and Workflows
4

80. Research Tools of the Trade
Our toolkit for ge ing survey’s done.
Typeform
Beautiful survey forms that display well
on web and mobile.
Surveys for market and user research
Qualaroo
In-app pop up survey tool with
targeting capabilities
Nudge or opt-in for a larger survey or
getting permission to follow up for a
another research study
Google Forms
Simple screeners and opt-in forms for
collecting people’s information.
Manage opt-in lists and screeners for
demographic data
Salesforce Pardot /
Gmail +
Boomerang
Marketing automation and email
marketing
Schedule and create emails for
distributing survey links. Manage opt-in
lists and research campaigns.
What It Is How It’s Used

81. Book Resource: Improving Survey Questions: Design and
Evaluation

82. Questions?
Chuck Liu
UX Research Lead
KISSmetrics
@chuckjliu
cliu@kissmetrics.com
Thue L Madsen
Marketing Associate
KISSmetrics
@thuelmadsen
tmadsen@kissmetrics.com

83. THANK YOU
Chuck Liu
@chuckjliu