Exploratory user research (How to figure out what to test)
•
5 likes•871 views
A look at how we conducted open, exploratory user research to figure out how we could make significant jumps in conversion rates at CareerFoundry. Analytics and tests only get us so far, but to have a bigger impact we need to explore, and learn something totally new. Something that we don't know we don't know.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Exploratory user research (How to figure out what to test)
Similar to Exploratory user research (How to figure out what to test) (20)
Recently uploaded
Recently uploaded (20)
Exploratory user research (How to figure out what to test)
- 1. @DhyanaScarano Exploratory Research How to figure out what to test
- 2. @DhyanaScarano • To have a drink and meet some cool people? • To learn some new methods and tools? • To learn about conversion optimisation and heuristic analysis? • To hear how other people do explorative research? • Something else…? Why are you here?
- 3. @DhyanaScarano Who am I? • Senior UX Designer at CareerFoundry • Co-organiser of Service Jam Berlin • Co-organiser of UXcamp Europe • Head mentor (product strategy), Google Launchpad startup bootcamp • Asks a looooot of questions Dhyana Scarano (Dee)
- 4. @DhyanaScarano • A bit of product strategy • A bit of research methods & tools • A look at a real exploratory research project underway at CareerFoundry • A little bit about how all that feeds into conversion optimisation What’s in this talk?
- 5. @DhyanaScarano • A little bit of product strategy • A little bit of research methods & tools • A hands on exercise (yay!) • Talk about all that feeds into conversion optimisation How does CO think about research?
- 6. @DhyanaScarano And what about Design Thinking? Research Underst- anding Define Testing ideas
- 7. @DhyanaScarano And what about Design Thinking? Research Underst- anding DefineProblem Testing ideas
- 8. @DhyanaScarano And what about Design Thinking? Research Underst- anding DefineProblem The real Problem Testing ideas
- 9. @DhyanaScarano And what about Design Thinking? Research Underst- anding DefineProblem The real Problem Solution Testing ideas
- 10. @DhyanaScarano And what about Design Thinking? Research Underst- anding DefineProblem The real Problem Solution Defining the problem Coming up with a solution Testing ideas
- 11. @DhyanaScarano And what about Design Thinking? Research Underst- anding DefineProblem The real Problem Solution Defining the problem Coming up with a solution Figuring out what to test Testing options Testing ideas
- 12. @DhyanaScarano And what about Design Thinking? Research Underst- anding Testing ideas DefineProblem The real Problem Solution Defining the problem Coming up with a solution (what we’ll talk about now) (Conversion optimisation) Figuring out what to test Testing options
- 13. @DhyanaScarano So, it always starts with a problem… ResearchProblem Problem: CareerFoundry needs to increase it’s conversions
- 14. @DhyanaScarano So, it always starts with a problem… ResearchProblem So how do we figure out what makes customers click “enrol”? Problem: CareerFoundry needs to increase it’s conversions
- 15. @DhyanaScarano So, it always starts with a problem… ResearchProblem So how do we figure out what makes customers click “enrol”? Problem: CareerFoundry needs to increase it’s conversions But wait. First, what does CareerFoundry do?
- 16. @DhyanaScarano So, it always starts with a problem…
- 17. @DhyanaScarano So, it always starts with a problem… How do we figure out what makes people click this?
- 18. @DhyanaScarano So, it always starts with a problem… How do we figure out what makes people click this? But then we need to find the “real problem” We can test different things based on what we already know We can also try and learn something totally new about our users that we don’t know yet!
- 19. @DhyanaScarano So, it always starts with a problem… How do we figure out what makes people click this? We can also try and learn something totally new about our users that we don’t know yet! Tweaking based on what we already know only gets us so far. We can test different things based on what we already know But then we need to find the “real problem”
- 20. @DhyanaScarano So, it always starts with a problem… How do we figure out what makes people click this? We can also try and learn something totally new about our users that we don’t know yet! New insights can lead to bigger jumps in conversion Tweaking based on what we already know only gets us so far. We can test different things based on what we already know But then we need to find the “real problem”
- 21. @DhyanaScarano ResearchProblem Original problem: CareerFoundry needs to increase it’s conversions How to do the research?
- 22. @DhyanaScarano Research Underst- anding Testing ideas em The real Problem Original problem: CareerFoundry needs to increase it’s conversions more people to click “Enrol” How to do the research? We can’t easily find the people who decided not to enrol with us. But we can ask the people who did choose us. Another challenge: our users are all over the world, from the U.S. to China, and even Australia!
- 23. @DhyanaScarano Research Underst- anding Testing ideas em The real Problem What was in their head just before they decided to click? What were they afraid of? Did they have any doubt or fears? What was the thing that helped them decide? How to do the research? What can we ask that will help us figure out what to put on our landing pages?
- 24. @DhyanaScarano Research Underst- anding Testing ideas em The real Problem What to ask? • We’re trying to discover new information, so… • Really open questions • Get them to tell their story How to do the research? How to ask it? • Need to explore and go deeper (interviews) • Also need enough data to draw conclusions (survey)
- 25. @DhyanaScarano How to do the research? The survey (For enough data) • Lots of free text responses, for thoughtful answers • Open questions. Who, Why, What, How, Can you tell us… • Asking them to tell the story of their decision • Try to discover things we don’t know we don’t know At the end of the survey we asked if they would be interested in doing a 45 minute interview for a $25 Amazon voucher
- 26. @DhyanaScarano How to do the research? The interviews (For deeper insights) • 45 mins • Field guide / conversation points • Only seen by the interviewer • Skype or appear.in used for video chat • Topics tailored to go deeper in to the survey questions
- 27. @DhyanaScarano How to do the research? The interviews (For deeper insights) • Recorded audio with a field recorder • Transcribed notes in Google Docs for easy sharing • Highlighted interesting quotes while typing for easy reference later Tip: Amazon vouchers were very difficult in multiple currencies. For the second round of interviews we didn't offer an incentive and tried personally messaging users. This took more manual effort but a higher percentage of users agreed to an interview.
- 28. @DhyanaScarano How to do the research? Scheduling interviews • Services like youcanbook.me and calendly.com let the interviewee book a time that suits you. • About 2 hours to set up and customise • Syncs with your calendar and only shows available time slots to the customer • You are notified when someone makes a booking, and it appears in your calendar
- 29. @DhyanaScarano Underst- anding Testing ideas Define The real Problem Solution • Frustration at current job Very high hopes in CF as a life changer • Fear of failure & lack of confidence • Time constraints biggest concern Spend more time than anticipated • Longing for community (also offline) UI & UX students much more so than WebDev and iOS General insights Understanding (the user)
- 30. @DhyanaScarano Course specific insights (e.g. WebDev) From interviews • Age: 33-53 (older than other courses) • Diverse professional backgrounds, often don’t live in a TechHub • Personal projects or business often a motivation to learn coding • Self starters with free code schools, need in-depth knowledge/skill polishing From survey • 80% interested in WebDev for some months or years • 60% considered a career path as a WebDev before signing up for the course • 40 % considered another CF course • 70% want to change career or job, some interested in freelancing Understanding (the user)
- 31. @DhyanaScarano New insights “I’m still apprehensive about posting on Slack [...] I’m worried that if I make a stupid mistake I’ll feel silly.” “ One of my biggest concerns was that after finishing the course my skills wouldn't be job ready. Since you added the job guarantee it instilled more confidence that the course is robust.” “My father was a computer scientist and he taught me how to code, I realized that I always liked to code.” “After reading the reviews, watching the YouTube channel and all the great feedback from the staff at CareerFoundry, I've no doubt that this will be the step I need to take.” Validation “I do have a small concern that my work will not be polished enough in six months to be able to successfully gain employment as a UI Designer.” Understanding (the user)
- 32. @DhyanaScarano Underst- anding Testing ideas Define The real Problem Solution Next? • Next we’ll update our user personas (different personas for each course) • Start to apply changes/tests to our home page, course landing pages and other parts of the service based on these new insights. • E.g. testing different value propositions based on what users have told us are their goals, doubts and fears. How does this fit into conversion optimisation?
- 33. @DhyanaScarano Research Underst- anding Testing ideas DefineProblem The real Problem Solution Defining the problem Coming up with a solution (what we’ll talk about now) (Conversion optimisation) Figuring out what to test Testing options
- 34. @DhyanaScarano Tools and acknowledgements • Surveys: typeform.com • Video interviews: appear.in (Click-and-go. No install required.) • Interview scheduling: youcanbook.me, calendly.com People harmed in the making of this work Tools • Clive K Lavery (freelance) Research lead, chief interviewer, insight gleaner • Jeff Humble (CF) Research support, transcriber, Amazon gift card victim • Claire Rackstraw (CF) Research support, transcriber, motivator