Suzanne Scharlock, Product Manager at Napster, gives us a behind-the-scenes look at how a PM handles UX and design issues. She talks through a project example to give you specific insights into her processes.
UX research at Napster: A Product Manager’s perspective
1. UX research at Napster:
A Product Manager’s
Perspective
#UTwebinar
@UserTesting
Suzanne Scharlock
Product Manager
Napster
2. 2016 Napster / Rhapsody International. Company Confidential
ABOUT ME
SUZANNE SCHARLOCK
Current: Product Manager at Napster. Building a new music streaming app.
Previous: Head of Product Marketing at Reveal Chat (formerly LikeBright).
3. 2016 Napster / Rhapsody International. Company Confidential
Global Addressable Base of 1 Billion Carrier Customers & CE Devices
3,400,000+
Paying Subscribers
90%
of Listeners use our
Mobile Apps.
>$140M
Royalties Paid in 2015
80%
Carrier
45%
Paid Subscriber Growth
250
Worldwide Staff
42,000,000
Songs and
Growing
20%
D2C
2014
2015
NAPSTER SNAPSHOT
4. 2016 Napster / Rhapsody International. Company Confidential
HOW DO YOU KNOW YOU ARE BUILDING THE
RIGHT PRODUCT?
4
5. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY
IDEATION
SURVEYING
USER TESTING
REVIEW AND REVISE RESULTS
IMPLEMENT
MEASURE
REVIEW AND REVISE RESULTS
6. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: IDEATION
PICK A KPI TO IMPROVE
Be specific with the user behavior you want to improve. Ex: 15% of users will create a playlist.
BRAINSTORM IDEAS
Ex: More users will create a playlist if the icon to open the playlist maker is on the homescreen
REVIEW PREVIOUS RESULTS
Look at previous data to find patterns of behavior where users are more likely to hit the KPI.
Ex: Users who open the PL maker multiple times in a session are more likely to make a playlist.
7. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: SURVEYING
CREATE HYPOTHESES
Write down assumptions and hypotheses. Write survey questions to incorporate hypotheses.
Ex: Users make playlists to listen to music
USE A SURVEYING TOOL
Run a survey using Amazon Mechanical Turk or Survey Monkey. Use surveys to understand
user desire and preference. Ex: When was the last time you made a playlist, what was it for?
SUMMARIZE AND DISTRIBUTE FINDINGS
Summarize the survey findings in a report and distribute the report to relevant people
throughout the company. Use the survey findings to shape the product requirements.
Ex: Users often make playlists based on mood and activity, incorporate this into feature
8. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: USER TESTING
CREATE PROTOTYPE / WRITE SCRIPT
Create a prototype and write a script for the test. Incorporate hypotheses in the testing script.
Hypothesis ex: Users can open the playlist maker button
IN-PERSON USER TESTING
Conduct in-person user testing with at least 5 members of the target market.
REMOTE USER TESTING
Conduct remote user testing with at least 5 participants on usertesting.com.
9. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: REVIEW AND REVISE
WRITE AND DISTRIBUTE USER TEST SUMMARY
Review all user test data and combine into a summary. Distribute summary throughout
company/team.Ex: Discover that participants did not discover Playlist Maker icon
REVIEW RESULTS
Review key learnings with product team members. Revise product requirements and feature
design based on learnings.Ex: Review summary with product team, discuss new icon choices
REVISE PRODUCT AND RETEST
Create new feature iteration and re-test either in-person or remotely. Ex: Devise new Playlist
Maker icon, create a new test
9
10. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: IMPLEMENT
CREATE HI-FIDELITY VISUAL DESIGNS/REDLINES
Work with visual designer to create hi-fi designs, redlines, icon fonts, assets etc.
Ex: Add new Playlist Maker icon to icon font, specify icon dimensions per device types etc.
WRITE FEATURE REQUIREMENTS
Create feature requirements including listing out KPIs for the feature and any needed metrics
requirements.
Ex: Write feature to swap icon, determine needed metric events for icon
REVIEW FEATURE REQUIREMENTS WITH DEV TEAMS
Review with development leads first to discover any holes in feature requirements, revise
feature requirements, then review with entire development teams.
Ex: Review icon change with dev teams
10
11. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: MEASURE
MEASURE KPIS THROUGH METRICS TOOL
Review KPI results on metrics tool such as Amplitude.
Ex: Measure open rate for Playlist Maker on Amplitude, determine if KPI was met
OBSERVE USER BEHAVIOR ON USERTESTING.COM
Upload mobile build to usertesting.com, write a script/hypotheses and verify that the feature is
usable in production.
Ex: Observe if participants open Playlist Maker via usertesting.com
11
12. 2016 Napster / Rhapsody International. Company Confidential
UX STRATEGY: REVIEW AND REVISE
REVIEW RESULTS
Review amplitude and usertesting.com results for the feature. Assess success or failure through
pre-determined KPI thresholds. Revise product if necessary.
12
13. 2016 Napster / Rhapsody International. Company Confidential
TEAM ORGANIZATION
PRODUCT TEAM
3 members 1 UX designer , 1 visual designer, 1 product manager
MOBILE DEV TEAMS
Includes: iOS developers, Android developers, QA testers, Program Manager
API AND PLATFORM DEV TEAMS
13
14. 2016 Napster / Rhapsody International. Company Confidential
TYPES OF RESEARCH
14
IN-PERSON USER
RESEARCH
Visiting students at a local college
REMOTE USER
RESEARCH
Usertesting.com
SURVEYING
USERS/TARGET MARKET
Apptentive, Mechanical Turk
15. 2016 Napster / Rhapsody International. Company Confidential
UX RESEARCH AND PLANNING
PRIORITIZE UX PROJECTS BASED ON KPIS
KPIs are organized into 5 Phases
1. Core Listening Use Cases
2. Premium Feature Use
3. Monetization
4. Onboarding Optimizations
5. Growth
15
16. 2016 Napster / Rhapsody International. Company Confidential
UX RESEARCH AND PLANNING
ORGANIZE FEATURE WORK
16
Feature/Update Screen or Flow Bug, Feature, or
Usability Update
Confirmed Work Rally Card Beta Priority
Group
Beta Priority
Description
Phase Platform Reasoning
Playlist
Maker
Playlist
Maker
Feature Yes
Link to rally
card
1.1
Listen to
Playlists
1-Implement
during beta
iOS, Android
Users create
playlists to
listen to
17. 2016 Napster / Rhapsody International. Company Confidential
PRODUCT/DEVELOPMENT KANBAN CYCLE
17
PO Backlog PO Definition Leads Review Meeting PO Refinement
Lead
Investigation
Queue of feature ideas
Queue of in-progress
feature ideas
Review features that
are product complete
with dev leads
Problems with features
discovered and features
are handed back to
product for review
Holes in features are
fixed by product team
and handed back to dev
leads for review
Team Review Meeting PO Finalization Ready to take Platform Napster API
Features are reviewed
with all client teams, QA
teams, and platform/API
teams
Problems with features
found in team review
meeting are fixed by
product team
Features are handed to
dev teams for
implementation
Features for platform
team
Features for API team
Client Queue Client Dev Testing Final Review KPI Review
Queue of features for
client team
Features currently in
development
Features currently in
testing with QA
Features are reviewed
by dev teams
Features are
implemented and
reviewed in amplitude
for performance
Completed Not Implemented
Feature is complete Feature is abandoned
18. 2016 Napster / Rhapsody International. Company Confidential
TEAMS WORKING TOGETHER
DELIVERABLES
Completed product requirements including feature rules and needed metrics, Hi-Fi visual
designs and redlines
18
19. 2016 Napster / Rhapsody International. Company Confidential
TEAMS WORKING TOGETHER
19
DEVELOPMENT TEAMS
Development teams expect a rationale behind “why” they are building a feature
MONETIZATION TEAM
Wants to know if a premium feature is desirable and usable
EXECUTIVE TEAM
Wants to know that product team is doing due diligence and creating features that will help
grow the company
20. 2016 Napster / Rhapsody International. Company Confidential
EXAMPLE: DOUBLE TAP TO LOVE FIRST SHOT
20
21. 2016 Napster / Rhapsody International. Company Confidential
EXAMPLE: DOUBLE TAP TO LOVE ROUND 2
21
22. 2016 Napster / Rhapsody International. Company Confidential
EXAMPLE: THIRD TIME’S THE CHARM
22
23. 2016 Napster / Rhapsody International. Company Confidential
USING USERTESTING.COM
USABILITY OF IMPLEMENTED FEATURES
Testing general features that have been implemented in the builds
COMPETITOR ANALYSIS
Testing usability of similar features
23
24. 2016 Napster / Rhapsody International. Company Confidential
USERTESTING.COM BEST PRACTICES
WRITE VERY SPECIFIC TASKS
Include warnings for users not progress to the next task until they completed the instruction for
the current task
USE SCREENING QUESTIONS
Ex: Testers must know how to turn airplane mode on their phone to be eligible for the test
ONLY DEPLOY A SINGLE TEST AT A TIME
Do a test run of your initital test to find any parts of the script that could have been confusing for
testers. Watch the test to the end and fix any hang-ups in the script before spending 5 credits
for a complete test
24