How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
Focus Groups are a Joke
1. 2013
Get up, get, get, get down
Focus Groups are a joke in your town
Source: http://upload.wikimedia.org/wikipedia/commons/5/57/Public_Enemy_4.jpg
Dan Berlin
Experience Research Director, Mad*Pow
dberlin@madpow.com
@banderlin
2. 2013
Why UXers should avoid focus groups
๏ Group dynamics
๏ Moderator bias
๏ Difficult to analyze
๏ Not going to reach consensus
3. 2013
Group Dynamics – Personalities
๏ David vs. Goliath personalities
๏ Dominant voice will dominate the session
๏ Timid voice may have wonderful ideas
4. 2013
Group Dynamics – Everybody Lies
๏ Participants want to seem more
interesting to those around them
Source: http://www.flickr.com/photos/kaykim/3649886264/
5. 2013
Group Dynamics – No Privacy
๏ Can’t discuss sensitive matters
๏ …and everything is sensitive
Source: http://www.flickr.com/photos/dannysullivan/4173190152/
6. 2013
Group Dynamics – Yawning!
๏ Yawning is contagious
๏ One yawn can kill a group’s momentum
Source: http://www.flickr.com/photos/supporttattoosandpiercingsatwork/8691355242/
7. 2013
Group Dynamics – Interrupting Cows
๏ Participants who interrupt others deprive
us from data
Knock knock.
Who’s there?
An interrupting cow.
An interrupting c…
Moo.
Source: http://commons.wikimedia.org/wiki/File:Cow_cartoon_04.svg
8. 2013
Moderator Bias – Experience
๏ Focus Groups are VERY reliant on a good
moderator
Source: http://commons.wikimedia.org/wiki/File:Leonard_Bernstein_3_Allan_Warren.jpg
9. 2013
Moderator Bias – Stamina
๏ A moderator must be cheerful and attentive,
listen closely, take notes, and plan the next
question for 2 hours.
Source: http://commons.wikimedia.org/wiki/File:David_Burlet_Friedrichsbau_variet%C3%A9_Plate_spinning_act.jpg
10. 2013
Moderator Bias – Trust
๏ Moderators must instill a sense of trust
with multiple people in a short amount of
time
Source: http://www.flickr.com/photos/anspach/954545/sizes/m/in/photostream/
12. 2013
Difficult to Analyze – Insights
๏ Almost impossible to quantify how many
people said something
๏ Hard to break down the data into
actionable insights
Source: http://www.texample.net/tikz/examples/escher-brick-penrose-t
13. 2013
Difficult to Analyze – Shallow
๏ You can’t get in-depth answers
Source: http://commons.wikimedia.org/wiki/File:Hilton_mug_shot.jpeg
14. 2013
Difficult to Analyze – Marketing Data
๏ We are UXers gathering Marketing data
Source: http://www.flickr.com/photos/justjenn/8546896516/
15. 2013
No Conclusions – Always reacting
๏ Participants are always reacting – they’re
not thoughtful
Source: http://www.quickmeme.com/meme/3v3d37/
16. 2013
No Conclusions – No consensus
๏ Not going to build consensus among
participants – leaves topics hanging
Source: http://www.flickr.com/photos/purplemattfish/3185323763/sizes/m/in/photostream/
17. 2013
No Conclusions – No one used anything
๏ We can gather how participants feel about
something and their out-of-context
desires…
๏ But we aren’t gathering real-world data
๏ It’s all made up… by our participants
18. 2013
Overnight Flight Example
๏ Focus groups said that they wanted
healthy snacks
Source: http://www.flickr.com/photos/gezellig-girl/3784584015
19. 2013
Overnight Flight Example
๏ Stewardess knew better and had
chocolate and sweets ready to go
Source: http://vintageseatcushions.deviantart.com/art/chocolate-fiend-154724164
20. 2013
Why UXers Should Avoid Focus Groups
๏ Group dynamics are terrible
๏ Moderator bias is inevitable
๏ Difficult to analyze marketing data
๏ You’re not going to reach consensus
So get up, get, get, get down
Focus Groups are a joke in your town
Editor's Notes
This will happen even if you have the best moderator in the world Irrelevant conversations, taking precious time
People lie by omission too
Get answers from everyone, talk about it a little bit, then move on