Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to ResearchOps Berlin Meetup #2 - UX Maturity - How to Grow User Research in your Organization
Similar to ResearchOps Berlin Meetup #2 - UX Maturity - How to Grow User Research in your Organization (20)
Recently uploaded
Recently uploaded (20)
ResearchOps Berlin Meetup #2 - UX Maturity - How to Grow User Research in your Organization
- 1. Welcome to our 2nd ResearchOps Berlin Meetup! ResearchOps Meetup Berlin PhotobyJoshuaLanzarinionUnsplash
- 2. Today’s agenda ResearchOps Meetup Berlin 6:30 pm Opening 7:00 pm Welcome and Intro: "UX Maturity Models" 7:20 pm "Fast forward with research" (FlixBus) By Luky Primadani, Katja Borchert, Carolina Schomer & Pietro Romeo 7:50 pm Q&A 8:00 pm Discussion in small groups 8:30 pm The End and Preview 10:00 pm We’ll all be kicked out!
- 3. Who are we ResearchOps Meetup Berlin Nikki UX Researcher, Instructor & Writer Anne UX Researcher, smart Helios GmbH Judith User & Innovation Researcher Lisa Service Design & User Research Katia Design Researcher & Strategist
- 4. Talk Name Speaker Name What is ResearchOps? "ResearchOps is the mechanisms and strategies that set user research into motion. It provides the roles, tools and processes needed to support researchers in delivering and scaling the impact of the craft across an organization.“ Kate Towsey ResearchOps Meetup Berlin
- 5. Talk Name Speaker Name Who is part of ResearchOps? ResearchOps Meetup Berlin User Researcher User Researcher doing Ops Research Ops Adapted from Emma Boulton
- 6. Why are we doing it 1. Sharing knowledge and experiences about user research 2. Getting to know each other on local & global level through different interactive formats 3. Improving the awareness of ResearchOps and its needs ResearchOps Meetup Berlin
- 8. What is a UX maturity model? ResearchOps Meetup Berlin
- 9. There are many out there. ResearchOps Meetup Berlin Design Research Maturity Model By Chris Avore @NASDAQ Design Design Maturity Model By InVision Implement user research to level up.
- 10. Talk by FlixBus ResearchOps Meetup Berlin
- 12. Discussion - in groups ResearchOps Meetup Berlin 25 min + 5 min overall sharing 1. Reflect on your company’s situation when it comes to reaching the next level of maturity in user research (if applicable). See the NASDAQ maturity model as example. a. Think about where you are at in terms of maturity level and rrite down your No1 current barrier for not progressing (3 min) b. Stick post-its on the wall and read the other post-its. (2 min) 2. (App. 2x): Someone starts sharing a barrier: Make it tangible - share a situation/story. (2 min) a. The others listen, then note down their suggestions on post-its. (3 min) b. Suggestions and best practices are shared in group. (5 min) 3. Each group shares a highlight/learning from their discussion in one sentence (5 min total)
- 13. Talk Name Speaker Name June ‘19 PREVIEW ResearchOps Meetup Berlin https://medium.c om/researchops- community
- 14. Talk Name Speaker Name June ‘19 PREVIEW ResearchOps Meetup Berlin User Researcher Skills Workshop
- 15. Links to all referenced material on UX Maturity ResearchOps Meetup Berlin ● https://inside.getyourguide.com/blog/2018/1/17/10-learnings-on-maturing-ux ● https://uxmag.com/articles/how-mature-is-your-organization-when-it-comes- to-ux ● http://vitor.io/evangelizing-user-research-enterprise ● https://medium.com/nasdaq-design/cultivating-more-research-more- frequently-ea202c32b0e3 ● https://www.invisionapp.com/design-better/design-maturity-model/ ● https://www.mckinsey.com/business-functions/mckinsey-design/our- insights/the-business-value-of-design ● https://uxdesign.cc/the-organizations-design-research-maturity-model- b631471c007c
- 16. Talk Name Speaker Name Let’s stay in touch. See you in June! ResearchOps Meetup Berlin
- 18. We are the new kids on the block TEAM SWARM
- 21. We support teams to test & validate their assumptions through user research, so that they can make more confident product decisions. Our team purpose
- 22. 1. Research at FlixBus: how everything started 2. The journey towards UX maturity 3. Fast forward: scaling research 4. How do we expect to reach the next stage Agenda
- 23. Research at FlixBus: how everything started
- 24. 2 years ago… ● No dedicated UX research & processes ● Product/design decisions based on gut feeling
- 26. Team set up was lacking of UX resources
- 27. Form the team & Build allies
- 31. The journey towards UX maturity
- 33. From awareness to adopting
- 34. From adopting to maturing
- 35. Fast forward in 2 years
- 36. From awareness to adopting
- 37. From awareness to adoption
- 38. Unmoderated remote study with employees Moderated onsite study with employees Moderated onsite study with customers
- 39. ● Meet-ups ● Conferences ● Internal knowledge sharing & UX Workshop ● Blogpost
- 40. Sharing UX Findings template Affinity mapShare videos
- 44. The demand for User Research has skyrocketed!
- 45. How can we run tests as EARLY, OFTEN and QUICKLY as possible?
- 46. More insights on cultural differences in different markets. How do customers use our products in their natural environment. Multiple studies at a time = no bottleneck due to complex recruitment and logistics processes. Remote testing tool
- 47. Remote testing tool - mixed methods studies
- 48. Democratizing UX is about people
- 49. Awareness Adopting Maturing Luky Luky Katja Pietro Carolina + Remote testing tool research partner Luky Katja Pietro Carolina + Remote testing tool research partner + POs, Designers, Devs, Stakeholders
- 50. Share the knowledge - PO Workshop
- 51. Divide and focus Marketing Apps CMS Ancillaries Booking Travel ExperienceSearch Internal Tools
- 52. Align with PO’s & their Roadmaps to prioritize
- 53. Share findings to generate discussions
- 54. When and how to do research?
- 55. Maturing and diversifying methods Awareness Adopting Maturing ● Usability Tests ● Guerrilla Test ● Usability Tests ● Guerrilla Test ● Remote Testing Tool (card sorting, tree test, click test, survey, think out loud) ● Focus Group ● Design Sprint ● Usability Tests ● Guerrilla Test ● Remote Testing Tool (card sorting, tree test, click test, survey, think out loud) ● Focus Group ● Design Sprint ● Field Study
- 56. Test prototype, website, apps User Labs: Think out loud interviews
- 57. Collect different opinions Focus Groups: moderated talks
- 58. Guerrilla Tests: surprise effect Quick validate user behavior
- 59. Design Sprints: collaborative work Solve a strategic problem - came up with a tested solution
- 60. UX Croissant: speed dating style userlab Quick Findings or Confirmations for small studies
- 61. Research timeline: testing the “problem”, not the “solution” *inspired by a model from “Validating Product Ideas Through Lean User Research” book by Thomer Sharon STRATEGIZE EXECUTE EVALUATE
- 62. Use cases
- 63. Tip the driver Self Check in Machine Mobile Web Redesign What do people want/need? What is the workflow? How people use the product? FlixBus use cases in the product development cycle STRATEGIZE EXECUTE EVALUATE
- 64. 1. Self check in Machine How do people use the product? USE CASES
- 65. How can we improve the Check in Process? Testing new Technology: Self check in machine Usability test How people use the product? STRATEGIZE EXECUTE EVALUATE
- 66. Qualitative research Experience prototype Fake Bus Passports Tickets Luggage “Driver” Focus Groups Discuss the experience After ride Survey Feedback questionnaire
- 67. Impact: product adjustments to diminish user misunderstandings "I think the machine doesn't like me" E., 65 years old
- 68. USE CASES 2. Tip the driver What is the workflow? How do people use the product?
- 69. Iteration: Usability test A/B test What is the workflow? How people use the product? How to improve bus driver’s quality and behavior? What is the better workflow for users to tip? Market research Qualitative research STRATEGIZE EXECUTE EVALUATE
- 70. Market Research: where should the feature be launched? Cultural behavior impacts acceptance
- 71. Qualitative research Guerrilla Testing Remote Unmoderated UserLab (customers + driver) Remote Moderated (driver)
- 72. Impact: Decide on the workflow & improve the usability issues on the prototype
- 73. 3. Mobile web redesign What do people want? How do people currently solve a problem? USE CASES
- 74. How users are currently using FlixBus mobile web? (Analytics + user research) Design Sprint Iteration: Usability test A/B test What do people want? How currently people solve a problem? STRATEGIZE EXECUTE EVALUATE
- 75. Impact: new redesign page has increased the CVR
- 76. How do we expect to reach the next stage?
- 77. How to reach the next stage 1. Field Studies to discover new findings in the real environment 2. Testing in different markets 3. Pro-active studies 4. Improve data accessibility with a “Results sharing platform” 5. Enable others to do research → democratize research 6. Exchange ideas with Research professionals > Best practices
- 78. Get in touch with us via LinkedIn or email: Luky Primadani Katja Borchert Carolina Schomer Pietro Romeo userlab@flixbus.com TEAM SWARM
- 79. Questions?
- 80. “Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford” Why You Only Need to Test with 5 Users by Jakob Nielsen “The goal in sampling with qualitative data is to detect repeating patterns— not to achieve statistical significance. With these smaller sample sizes qualitative research looks for signals, which are indicators of trends within a population” Kelly Moran, Google UX Researcher “With qualitative research, you’re zooming in and getting a closer picture of fewer people. So that you know them better” Kelly Moran, Google UX Researcher What is the relevance of a study with only 5 people? Is this sample size statistically significant?
- 81. 4. Branding app icon Which design generates better results? USE CASES
- 82. Which app icon made people download the app the most? What is the brand perception in new markets? Internal designers create logotype icon option Survey Click Test A/B Test Which design generates better results? New Icon: Logotype Current Icon: Logogram STRATEGIZE EXECUTE EVALUATE
- 83. Qualitative research Click Test Remote Unmoderated Survey
- 84. QUALITATIVE RESULTS: The new icon improves usability and brand image. It is easier to find and more modern. QUANTITATIVE RESULTS (A/B Test): Install rate uplift of ~5% on Google Play. Impact: change for the Logotype icon
Editor's Notes
- Photo by Joshua Lanzarini on Unsplash
- Also: Mention house keeping rules, who is taking pics and should be informed if someone doesn’t want to be in the pic
- (For Chart 7: There is a growing global community since about a year with an active slack, medium and online town halls. All over the world workshops happened to define the needs of UX researchers and define research operations. In June, Chris and Kat organized a workshop in Berlin (at Here maps). Since then, we’ve met 3 times in a smaller group but felt there’s a need for a regular, public meetup focusing on user and design research. Intl. community, No one owning it ))
- Ops- What? “Everything is ‘ops’ today…”! “Not another Ops” Yes, because research demand is growing, but it lacks resources and infrastructure. So R+ is about the infrastructure, e.g. recruiting, knowledge management (like data repositories), governance, tools etc. It’s everything that supports research/the craft, implementing research. / research about researchers
- Who is in the community? Looking at the slack channel conversations and the current state of Research this only makes sense. We have very few people soley doing Research Ops hence there is such a huge overlap of user researchers also doing Ops related tasks and activities. // Anyone mainly doing ResearchOps? Who is mainly working as a user researcher? Who is partly doing user research (e.g. UX Designer, Service Designer)? Who is usually less involved in user research (e.g. PMs)? / meet up not restricted to pure Ops topics - and boundaries are blurry
- Tonspur auf S 7
- mention the Chatham House Rule: “...participants are free to use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of any other participant, may be revealed.”
- What is a UX maturity and Why could it be helpful for us? I think this is why you are all here tonight. You want to find out what UX maturity is and what kind of models are out there and what this could me mean for you.
- There are many models out there. The all indicate where an organisation exist in a spectrum across levels of maturity AND they allow us to identify where to take action and where to improve. It is important to know, however, that the different models depict maturity from different perspectives such as Design Team (which only at some point includes user research) and others specifically address User or Design Research maturity only such as the one by Chris Avore. Left to right: 1) Maturity Model by Invision (5 Levels, Desgin Team, Research is included at Level 3; Majority of Companies is 1-3) 2) Design Research Maturity Model (4 Stages, Research Specific, lists many attributes concerning organisational set-up) 3) Third one will be introduced in more Detail in the Talk by Flixbus (Based on roughly 2.2000 designers) http://www.invisionapp.com/design-better/desing-maturity-model ResearchOps relevant only up from level 3 → Level 5 makes roughly 5% Generally the following characteristics have these modelsl in common independent of number of steps and activities: Aesthetics/design/UI, not that many people involved, low awareness More people involved, workshops, usability Dedicated team, bigger audience training offered, research becomes democratized, tied to KPIs and revenue Whole org aware of it/fully integrated, research for strategy & discovery, organized, dedicated ResearchOps role
- We wanted to focus on Chris’ modul as we thought it might be easier to locate yourself as User Researchers or Research Activities you are aware of in your organisation. Print outs are in various rooms spread across the rooms.
- prepare whiteboards // give people a chance to leave/step back in case they don’t want to be part of the discussion Use the watch Randomly assign participants and ourselves (moderators) to groups (1,2,3,...up to 8), if too homogene people might switch Short clustering in groups could be an option so that a barrier that many people experience is tackled. It is not required that people put their organization into a certain maturity level but get a general impression of where they are standing. Pay attention to these sub-topics (esp. if discussion is fading): middlemen, sponsors/supporters in your organization, working at an agency vs. corporation, informing strategy vs. product, top-down vs. bottom up (Is it better to implement UX maturity throughout the whole organization (top down) or set it up individually at each product team (bottom up)?) Scaling, training, KPIs
- We (dedicated ResearchOps team) have reflected upon where our organisations stands in terms of UX maturity. Yet, it is also important to think about our on skills set. What level are we at? What is ahead of me. We want to provide the opportunity to answer these kinds of questions. Hence we trial running a skills workshop as the next meet up format. // There will be skill sorkshops happening across the globe. It is an initiative of the ResearchOps Community.. We’ll provide more information and links on the Meetup page. // Sign up is free and first come first serve basis
- Takes about 2.5 hours // We’ll provide more links on the Meetup page. / Reach out if you want to talk or host // @Anne, @Luky, @Carolina, @Carina, @Judith
- Slides will be put on slideshare, recording…
- Thank you, thanks FlixBus!
- I want to quickly introduce our team to you Our Cultural Background is as different as our Professional backgrounds Luky comes from Indonesia and is a experienced UI & UX Designer I’m from germany with background in Product Management Carolina from Brazil she worked many years in Branding and Marketing Pietro from Italy with background in Computer Science To give you a quick outlook: Luky is going to talk about UX in general & evolution of UX here at FlixBus, I will tell you a bit about how did we scale research Carolina & Pietro are going to present a few interesting case studies After our presentation we are happy to answer your questions
- When we want people to learn something, we want them to particiopate
- Ux mindset
- So until here Luky explained how do you convince people that UX is important and the whole strategy to make the company and the stakeholders adopt it. So I’ll be taking you through what happened since then. How do we set up the team to attend the requests and accelerate the research adoption.
- The 1st thing is that the demand for research increased. More areas saw the benefit of it and wanted to try research. Teams were talking and recommending the SWARM team.
- It’s not just about tool, democratizing is about involving people
- This is how we see people in the stages. Research Partner. Empower more people and have a Facilitation role.
- Example. POs. Gurrilla Test (convenient and phrase questions).
- Pushy → Prioritizing.
- In this stage we still need to make the findings access and generate discussions
- What we Understand - more mature we get, the topics also get more strategic, we need to combine more tests to get deeper findings
- Deep interview: in depth interview
- Involve people → commitment and sense of belonging
- Finish my part of scaling research, this is a timeline we based ourselves to help us organize and also understand the studies we are dealing with. What is clear for us is that maturity is really about testing the problem and not the solution. So think this is a product development timeline. When we test in the begining
- Interesting case because it involved four product teams and two different user segments - customers and bus drivers. There were A LOT of people involved in this project and hence it required a lot of different research studies.
- Google Play AB-Test Results: The new icon leads to more installs Germany: Install rate uplift of up to 4.3%, which means up to 16.7K more installs on Google Play only in December 2018 (potentially another ~ 10K on iOS) US: Install rate uplift of to 7.4%, which means up to 6K more installs on Google Play only in December 2018 (potentially another ~ 4K on iOS) The new icon called more attention (+clicks) The new icon was easier to find and more associated with the brand While the original icon (arrows) is perceived as more dynamic, the new icon is perceived as more digital/modern