5 tips for user research ppt
•Download as PPT, PDF•
16 likes•14,404 views
The slides from my O'Reilly Webcast on 8/28.
Recommended
More Related Content
What's hot
What's hot (16)
Viewers also liked
Viewers also liked (20)
Similar to 5 tips for user research ppt
Similar to 5 tips for user research ppt (20)
Recently uploaded
Recently uploaded (20)
5 tips for user research ppt
- 1. @lauraklein Essential Tips for Lean User Research
- 2. @lauraklein Who is this for?
- 3. @lauraklein This Is For • Entrepreneurs • Founders • Engineers • Product Managers • (Some) Designers
- 4. @lauraklein Don't Just Sit There
- 5. @lauraklein Tip #1: Know What You're Testing
- 6. @lauraklein First An Exercise Write down the top thing you'd like to learn about your product or market you want to test. Go on. I'll wait.
- 7. @lauraklein Are You Asking... ...what or why? ...about your user or your product?
- 8. @lauraklein Why Should You Care? The type of question you ask determines the type of research you should do. What == Quantitative Why == Qualitative User == Ethnography Product == Usability
- 9. @lauraklein Tip #2: Know Who You're Testing
- 10. @lauraklein Hey Look, An Exercise! Write down a quick description of who you're building your product for.
- 11. @lauraklein Did You Write "Myself"? If you did, I hope you're independently wealthy or that you have a lot of clones.
- 12. @lauraklein Pro Tip You are not looking for "moms". *also, not "pet owners", "doctors", or "salespeople".
- 13. @lauraklein Be Specific "The person using my product owns more than one electronic device on which she has tried to view documents for work. She finds this difficult & frustrating."
- 14. @lauraklein But How Do You Find Them? Friends of Friends Social Networks Professional Recruiters Blogs & Forums Appropriate Physical Locations Email Lists
- 15. @lauraklein Why Should You Care? If you don't know who you should be testing, you don't know who you're building your product for.
- 16. @lauraklein Tip #3: Know Which Corners To Cut
- 17. @lauraklein Need Some Exercise? Quick, write down as many different types of user research you can think of that take under an hour of your time.
- 18. @lauraklein Less Than 1 Hour • Remote Usability with UserTesting.com • Landing Page Test with Unbounce • Google AdWords Test (provided you have an account) • Remote Observational Session with a Current Customer
- 19. @lauraklein Less Than 15 Minutes • Five Second Test • Click Test • Friends & Family Usability Test • Guerrilla or Hallway Usability Test
- 20. @lauraklein Why Should You Care? Not all research is this fast, but if you can answer a question in 15 minutes, you have no reason for not testing.
- 21. @lauraklein Tip #4: Know What To Do With The Results
- 22. @lauraklein Yet Another Exercise I want you to write down what you typically do after a user research session.
- 23. @lauraklein Ok, here's what you could be doing
- 24. @lauraklein Gather Observations Have the moderator and observers write down 5 key observations or problems they observed.
- 25. @lauraklein Sort Them By Metric Revenue Retention Activation
- 26. @lauraklein Stack Rank Them Put them in order of estimated importance. Hang on to these stickies, because you'll iterate after every session.
- 27. @lauraklein Other Methods 1. Debrief after every session 2. Capture the info in a persistent way 3. Determine the most important lessons 4. ACT ON THEM
- 28. @lauraklein Why Should You Care? Having a familiar way of aggregating data after any individual test helps you understand the bigger picture being created by your research.
- 29. @lauraklein Tip #5: Know The Unanswerable Questions
- 30. @lauraklein Last Exercise! Write down a question you've asked in research that users couldn't answer.
- 31. @lauraklein Was It One Of These? • Would you buy this? • How much would you pay? • How would you design it differently? • What new feature would you like most?
- 32. @lauraklein Answerable Questions • How else have you tried to solve this problem? • How have you found similar products in the past? • Walk me through exactly what happened the last time you used this product.
- 33. @lauraklein Why Should You Care? Unanswerable questions make you think you know something. But you don’t!
- 34. @lauraklein More Questions? You know where to find me... laura@usersknow.com http://www.usersknow.com UX for Lean Startups
Editor's Notes
- Hi. Welcome. Thanks for joining me today. We ' re going to talk about some Tips for Lean User Research. This shouldn ' t take more than a half an hour or so, and then there will be time for questions afterward.
- I always like to start things off by making sure that everybody is in the right webinar so let ' s look at who this is for.
- Specifically, this talk is for people who make stuff for a living. It doesn ' t assume that you know anything at all about user research beyond the fact that you ' re supposed to do it. If you ' ve tried learning things from users before or you ' ve been at a company where it ' s been done, that ' s fine. I ' m not going to make you leave or anything. But you ' ll notice that professional researcher isn ' t on this list. This isn ' t for them. If you do research for a living, you should already know all this because we ' re going to talk about some really important, really basic stuff. If you ' re already getting out of the building and talking to people, this talk should make you better at doing that. It ' ll get you thinking about the right questions to ask before you start asking your questions of your users.
- I ' m going to require just a little bit of work from you. I ' m going to have you writing a few things down, so it might be a good idea to get a pen and paper ready or set things up so that you can take notes on your computer. Instead of just drifting off to the not so soothing sounds of my voice, I want you think actively about whatever product you ' re currently working on. When you leave, I want you to be able to immediately go out and start using what we talk about here today.
- For our first tip...Know what you ' re testing! Well, this one seems obvious. I mean, before you go test something, you should know what it is, right? Stick with me. It ' ll make sense in a few minutes.
- Ok, I want you to write down the most important thing that you want to learn about your product or market right now. This is a thing you might try to run some sort of experiment or conduct some sort of research to learn. Don ' t think about it too hard. I ' m only going to give you about 20 seconds to do it. ... Got it? If not, we ' re moving on anyway.
- Now, I ' d like you to look at the thing you ' re trying to learn and ask yourself these questions about it. Is it a " what " or a " why " question. What questions are things like " What step of registration are my users getting stuck on? " Why questions are, " Why are users getting stuck on step 3 of registration? " See the difference? Another thing to look at is whether you ' re learning about your user or about your product. A user question is, " How do people in my persona group currently shop for groceries? " You ' d obviously only ask something like that if your product was somehow grocery related. A product question is " can users find a particular type of grocery item using my product? "
- So, why should you care if you ' re looking at Why or What research or User or Product research? The reason this is important is that what questions can often be best answered using quantitative measurement while why questions should be answered with qualitative research. Similarly, User questions can often be answered with ethnographic methods, where you actually go out and watch people ' s natural behavior, while product questions are often easier to answer with usability research. That ' s the kind you may be familiar with where you give people specific tasks to perform using your product. The thing that I see all the time is people thinking that there is only one kind of research they need to do - that interviews or usability tests are the sum total of research. That ' s just not true. I won ' t have time today to go into the million different types of testing, both quantitative and qualitative, you can do, but once you know which type you ' re looking for, you can do a deeper dive into how to conduct specific types of tests. also, I ' ll give you my email at the end of this talk, and you can always write and ask me specific questions. I really do respond to all emails. Eventually.
- Ok, Tip #2. This, I ' ll admit, also seems somewhat obvious. But you ' d be surprised at how many people get this wrong. We get told to get out of the building a lot, but then we ' re not always told where we should go. Or we get told to " talk to customers " but what if you don ' t have customers? Or what if your customers are hard to get ahold of? Or what if you ' ve talked to them already?
- Let ' s do another quick exercise. I ' d like you take a minute and write down the description of the person you ' re building your product for. I don ' t want their age or how many pets they have (unless this is somehow relevant to your product). I want a description of what sort of person would use your product stated in a way that makes it clear why they ' d use your product. I know, it seems a little confusing, but take one minute and give it a shot. We ' ll get into specifics in a minute.
- Ok, first question. Are you your user? NOPE! I mean, you may be one of your users, but unless you ' re literally building a product aimed at first time startup founders of angel funded companies with fewer than 5 people (or whatever fits your particular exact description), you ' re not the target. Hell, even if that IS your market, you still know way too much about your own product to be a valid use case.
- Here are some other things that people say when I ask them who their persona is. Moms! But " moms " is not a valid persona. Why? Think of five moms you know. Maybe you have one, for example. Maybe you are one. Maybe you ' re married to one. Maybe you went to high school with one. A single mother of three who is working two jobs and lives in the city can ' t be in the same persona group as a stay at home mom of one who lives in the suburbs. They may both have kids, but they have different problems, and trying to build something to make them both happy is really, really, really hard. At least at first. Remember, even Facebook started off as just Harvard students.
- Here ' s a better example. Let ' s read that together. " The person using my product owns more than one electronic device on which she has tried to view documents for work. She finds this difficult and frustrating. " Note that for this particular product I don ' t mention things like how many kids she has or where she lives. That ' s because, for this particular product, those things aren ' t relevant. What ' s relevant is that it ' s a person with multiple electronic devices, all of which are used to perform work, and that the work involves some sort of documents. I also noted that this is a problem. Since, you know, Dropbox wouldn ' t be as popular as it is if this particular task had been easy for everybody.
- Of course, once you have a description of the people you ' re interested in talking to, you have to find them. I get asked for suggestions on this a lot. These are some of my favorite places to recruit research participants. I ' ll often create a short, qualifying survey called a Screener and post that on the web. Then I ' ll drive traffic to it by emailing friends or posting to Twitter or commenting on blogs or forums. In some cases, I recommend that founders build an email list or blog first, before ever building a product, since this allows them to start gathering an audience that can then be mined for user research. I mean, you have to build your user base anyway. You might as well get as much of it done as you can before you build your product.
- But why does it matter who you research with? Why can ' t you just go to Starbucks and show off your awesome prototype and see what people think? Well, if you just want to do some simple usability testing - for example, maybe you want to know if a normal person could get through your registration flow - then guerrilla usability testing at Starbucks is a fine way to go. But if you ' re trying to understand the person you ' re building the product for or you ' re trying to learn what problems people might pay you to solve, you have to go to the source. You can ' t figure out if your product for astrophysisists will be useful by talking to some folks at Starbucks. Unless that Starbucks happens to be the one right next door to NASA. Then you might have a shot.
- But Laura, you ' re saying. This all sounds so haaaard. Isn ' t there something we can do to make this a little faster? Or cheaper? Ok, fine. I ' ll teach you some tricks. But I want to say right now that these cannot be the only types of tests you run. Sometimes you have to buckle down and really have in depth conversations with real people in your persona group in order to truly understand their lives and problems. But sometimes you can cut a few corners.
- But first, I ' m going to make you do something again. I ' m just making sure you ' re still awake. This time, I want you to take 30 seconds and write down different types of user research you can do in under an hour. As many as you can think of. This is an hour of YOUR time, by the way. A few of the ones I mention will take well under an hour to set up, although they might have to be left alone for a bit once you get them started. Ok, go.
- Did you get any of these? Because you can start a research session with any of these in under an hour. Of course, for some of them, you ' ll need to wait a bit to get results back, but that ' s time you can spend doing other things. So, why not take an hour out of your day and start a remote, unmoderated usability session on a new feature or a badly performing registration flow? That last one is interesting, because you can often run the whole thing in under an hour. If you have some access to current customers, it takes well under an hour to send an email, set up a time, and then have a conversation or do a screenshare with a current customer. In fact, it ' s great to have a roster of real customers who are willing to jump on GoToMeeting and take a look at a new feature or even a wireframe to give feedback.
- But did you know that you can also get feedback in 15 minutes? Now, granted, the types of feedback you can get are severely restricted. You ' re not going to do great customer development in this amount of time, but you can absolutely get good usability information. One of my favorite, less well known ways to get feedback is the 5 Second Test. There ' s actually a website called 5 Second Test although you can do this in person too. You upload a mockup or picture of a landing page and write a few questions you ' d like people to answer. Then people who come to the site are shown your landing page for 5 seconds and asked to answer your questions about it. I like to ask questions like, " What does this product do? " and " Who is this product for? " The reason this is so powerful is that it tells you what complete strangers think your product does. Try it a few times, and you ' ll be surprised at how often the messaging on your landing page is failing to convey that particular piece of information. It is simply the best way to figure out why your messaging is failing, and it can provide wonderful clues about why people aren ' t converting.
- You ' ll notice that a lot of the research that can be performed this quickly is some variant on Usability testing. You ' ll get a lot of feedback on whether people can easily perform some task or other in your product. It ' s hard to cut corners on real customer development or understanding your market. But this is still extremely important information! Learning that your registration flow is incredibly confusing is the first step to building a registration flow that is not incredibly confusing. I ' m always shocked at how many startups don ' t even do the small amount of work required to make sure that people can use their products.
- You know what ' s worse than not doing user research? Doing it and then not doing anything with the results. This is bad because then you ' ve sunk time and money into something that you completely ignore. Sometimes this happens not because people willfully want to ignore results. But often happens because people have absolutely no idea what to do with all the data they collect. So, let ' s look at a way to fix that.
- Let ' s do another quick thought exercise. I want you to write down what you typically do in the 5 or 10 minutes after you ' ve talked to a user or a potential user. I ' ll give you a minute.
- Ready? Let ' s do this.
- Let ' s say you ' ve just ended a research session. You ' ve talked to somebody for 30 minutes about your product or mockup. Hopefully you ' ve had more than one person in the room with you. Trust me. These things go better if you have at least one person to lead the conversation - that ' s the moderator - and one person to take notes. Feel free to add a third if it doesn ' t make the participant uncomfortable or if you ' re doing this remotely. Take a stack of post it notes and divvy them up. Spend a few minutes writing down 5 key observations or problems that each person saw. Do this independently! This is not a group think exercise. Part of the point of this is to make sure everybody on the team is hearing the same thing, and you ' ll do that better by writing down your observations independently and then comparing later.
- Next, I want you to think of all those observations as potential fixes or features and ask yourself what metric would change if you changed them. For example, let ' s imagine that you noticed people had trouble finding the next button on the registration flow. If you were to fix that somehow, presumably your registration metric would improve. On the other hand, let ' s imagine that you ' re noticing that people who have used your product for awhile tend to stop coming back after week three. If you were to figure out why and fix that, it would presumably improve your retention metric. What ' s that? What happens if you notice something that won ' t materially change an important metric? It won ' t make you more money or get people to come to your site more often or visitors to regular members? Why exactly would you want to make a change that ' s not likely to improve a key metric? Anyway, it ' s up to you how you define your key metrics. I can ' t cover that in this talk, but the book Lean Analytics will help you figure all that out. The important thing is that you sort all of your changes or feature ideas or bug fixes into the metric you think that they will affect most. I know this sounds confusing, but it ' s really much easier than you think once you start doing it.
- Once you ' ve got everything sorted by metric, stack rank each column. Try to figure out the highest priority things to change first. Now, this is going to be subjective, but you ' re looking for things with the highest return on investment. Ok, this is actually the hardest part. It ' s tough to estimate whether fixing the next button on registration is going to really make a bigger difference than adding a brand new feature that encourages more people to register. The thing is, you ' re going to get this wrong at first. But if you continue to go through this process and measure your actual outcomes, you ' ll get better at it over time. You ' ll never be perfect at estimating ROI, but I ' ll tell you right now that people who do this over and over get better at it quickly.
- Ok, I ' ll be honest. You don ' t have to follow this exact model. There are other ways of doing this. But the important parts of the model are: 1. Debrief after each time you run an experiment or talk to a user 2. Capture that information in some sort of persistent way 3. Determine which are the most important things to act on and 4. ACT ON THEM Again, the only worse than not doing research is doing it and then not benefiting from it.
- So, why should you bother with this? Well, by having some standard way of dealing with information after you run any sort of test, whether it’s a customer development interview, a usability test, or even an a/b test, you’re going to be more likely to understand the big picture. You’ll be able to see which things come up over and over again in research. This is important, because sometimes it’s really easy to listen to the loudest person rather than hearing things said quietly by lots of people. You’ll also be able to make connections between quantitative and qualitative research more easily if you’ve got persistent data. For example, if you’ve done an a/b test and learned that one registration flow is better than the other, that’s going to make even more sense once you’ve run some usability tests on both branches of your a/b test in order to learn why a or b won. Having persistent, visual data from your tests also helps you communicate the results to other people in the company, which is important when trying to build consensus around which changes you’re going to make.
- Ok, last one. Perhaps the most important one. Know the unanswerable questions! If you ' ve ever talked to a user or a potential user, I almost guarantee that you ' ve asked one of these questions. Hell, I ' ve asked some of them. But you have to stop doing it. You see, there are questions that people simply can ' t answer.
- Here ' s your last exercise of the day. I want you to write down a question that you ' ve asked to a user that they couldn ' t answer. This is a quick one. I ' ll give you 30 seconds. Go.
- Was it any of these? No? What ' s that? You ' ve asked these questions and people have answered them? Yeah, they lied. I mean, they didn ' t lie on purpose. They weren ' t being malicious. They were probably even trying to be helpful. But the problem is that a user or potential user really can ' t answer these questions correctly. You ' re asking them to tell the future or to design your product for you. Neither of those are things that people are likely to be able to do. Let me give you a little example. Let ' s say that I were to ask you, " Are you going to eat a cookie tomorrow? " It seems simple enough. But you don ' t really know what ' s going to happen tomorrow. Maybe you eat a cookie every day for lunch so you think it ' s pretty safe to say yes. But what if you ' re sick tomorrow and can ' t leave the house? Maybe you ' ll drop the cookie before you get a chance to eat it. You don ' t know. In fact, just because I ' ve mentioned cookies today may subconsciously make you more likely to want a cookie. I ' ve put cookies into your head, and now you ' re thinking about them. I may have convinced you to eat a cookie that you wouldn ' t otherwise have eaten, which, it turns out, is not user research. It ' s sales. And it ' s dangerous if you ' re trying to figure out if people, left on their own, would buy your product. The design questions are even worse. You ' re literally asking the user to do your job for you. I mean, this is a product you work on every day. You think about it constantly. Maybe you built it from nothing. Why are you suddenly going to turn over the design decisions to somebody who agreed to talk to you for 45 minutes and who may use your product once a week? That ' s insanity. Do your job.
- What should you ask instead? You ' re better off spending your limited time talking to people about their problems and asking them about their current or past behavior. Ask them whether they ' ve looked for or even bought similar products in the past. Ask why they aren ' t using those products any longer or, if they are, what they love about them. Ask them how they found similar products, since this might help you understand your distribution channels. Ask them to tell you exactly what happened the last time they used your product and note all the things that they said were hard or that they had to use some other product to accomplish. That ' s how to ask the unanswerable questions in a totally answerable way.
- So, hopefully I ' ve made it clear why you should care about these questions and never ever ask them. But I want to point out one more reason why these questions are poisonous. They will make you feel like you know something. If somebody says that they will pay you $20/month for your product, you will feel like you know people will pay you $20/month for your product. But you don ' t. You won ' t know people will pay you until they ' ve handed over their credit cards or signed a legally binding document.
- Ok, that ' s it for my talk. I ' m going to take some questions from the audience now. I ' d like to keep the questions focused on lean user research, but if you have any questions about other aspects of Lean UX or if you want to ask really specific questions about your particular product or company, feel free to send me an email at [email_address] . If you ' d like further information on topics like these, please follow me on Twitter at @lauraklein . Also, if you enjoyed this talk, don ' t forget to * buy my book UX for Lean Startups * read my blog * take one of my workshops that are going to be offered starting soon at LUXr. that ' s the Lean UX Residency - L U X R. You can find out more about them at luxr.co. Also, I believe I have another webinar coming up in a little more than a month. In that one, I ' ll be having a Q&A with Eric Ries, who is awesome. Thanks again! Who ' s got a question?