1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris

Product Analytics 101
A introductory view on analytics for
product design, development and management
George @ Voulgaris .net
@gevou

How many products will you be selling in 12 months from
today?
George Voulgaris, Ph.D. - Product Analytics 101
1st Prod.Active MeetUp, Athens, Dec 10th, 2014

Short tale instead of intro
“Sales Forecast: The Quest for Holy Grail” - or - “the 50 shades of finance”

1. Tools

User Testing
What is the best homepage copy for driving sales?
Testing tools give you insight into how people are
using your product, so you can optimize your design.
Experiment with variations using A/B tests and
understand behavior with heat maps to see what
version works best.
Advertising
Which ad is driving the most sign ups?
Advertising tools allow you to add event
tracking and ad retargeting to better
understand your site visitors, where they came
from, and if your campaigns are working.
Analytics
What are customers doing before they churn?
Analytics tools allow you to visualize your data,
see how customers use your product, and
identify trends, so you can make better
business decisions.
Marketing Automation
Did that newsletter send traffic to your site?
Email marketing tools make it easy to send
newsletters and other marketing materials to your
customers. Integrating with these tools makes it
easy to setup drip campaigns for your product.
Sales & Business
What sales leads and customers need the
most attention today?
Sales & business tools help you track leads
throughout the sales funnel, manage customer
relationships, and run outbound campaigns.
Customer Support
How can I answer any questions my customer
have?
Customer support tools help you communicate
with current customers to answer their questions,
provide support, and identify recurring issues.
Developer Tools
Why couldn’t new users sign up for accounts
yesterday and how much money did we lose?
Performance and error reporting tools help you
improve the health and stability of your application by
automatically logging and tracking errors and alerting
you when there is any downtime.
source: segment.com

few examples

inspectlet.com

inspectlet.com
Replay user activity* for the ultimate insight
on how users actually use your app
*be careful with your T&Cs and the possibility of sensitive data

mixpanel.com

Add Push notifications, Surveys, Revenue analysis…
mixpanel.com

… + pretty much any data you want to send from your App
mixpanel.com

Advanced Segmentation, In-depth profiling,
Direct Engagement / Communication
intercom.io

intercom.io
… combined with automated messaging.

“Q: Is this the:
a) best
b) recommended
c) endorsed
d) cutest
set of tools?”
A: These are just examples. I do not endorse any - I just
happen to use them at the moment. Do your own research
for best fitness to your case.

With all those beautiful apps, tools, analyses,
graphs, functions, automations you should be
able to build the perfect products and
businesses!
Right?

What tools do you use? (or fooling around with)
How and Why?
What are your challenges at the moment?

2. What matters?
(because observation is not a business by itself… mostly)

why

There is a great difference in focus for startups and established
businesses. What is it?

There is a massive difference in focus between Customer
Discovery and Customer Development stages. What is it?

where

The Long Funnel gets even longer
Customer engagement inc.
churn, retain, upsell
Inbound
Marketing
website &
purchase funnel in product
UX
Customer
Support
Company

Inbound
Marketing
website &
UX
Customer
Support
Company
Layered Inbound Marketing diagram source: Inbound Marketing Handbook - MarketingSherpa

Inbound
Marketing
Sales & Business
User Testing
website &
UX
Customer
Support
Company
Advertising
Analytics
Marketing Automation
Developer Tools Customer Suppor
BI
Layered Inbound Marketing diagram source: Inbound Marketing Handbook - MarketingSherpa

when

Which stages of the entire journey / funnel are you
currently monitoring in your business? How?

What matters most depends on the product / business maturity
stage and the user engagement stage.
There are few frameworks that can help.

Dave McClure’s Pirate Metrics
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
Acquisition
Activation
Retention
Revenue
Referral
Source: http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version

Lean Analytics
(A.Croll & B. Yoskovitz recommended reading)
Gate
Stage
EMPATHY I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE The users and features fuel growth organically
and artificially.
SCALE I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
The five stages

Both AARRR & Lean Analytics frameworks resonate with…
Eric Rees’ (The Lean Startup)
Sustainable growth is a requirement for any business.
“New customers come from the actions of past customers”
The three growth engines:
Virality
Make people invite others as a necessary side effect
Rate of growing
Stickiness
Keep people coming back / addictive / network effects / lock-in
Rate of churn (compound)
Price
Reinvest revenue of past to get new customers.
Rate of expenditure

how

Lean Analytics
(Adapted version from the book by A.Croll & B. Yoskovitz recommended reading)
Pick a KPI
Draw a line in the
sand
Identify ways to
improve this KPI
Make a guess Find relevant
patterns in your
user data
Give up or pivot Go back to
Design
experiments
(e.g. A/B)
Line-in-the-sand
was crossed
Implement what
seems to work
Find correlations
between those
patterns & users
data
available
no data
Analyze the
behavior of the KPI
Line-in-the-sand
was NOT crossed
Discovery.
Talk to customers
Try again

Lean Analytics
(Adapted version from the book by A.Croll & B. Yoskovitz recommended reading)
Pick a KPI
Draw a line in the
sand
Identify ways to
improve this KPI
Make a guess Find relevant
patterns in your
user data
Give up or pivot Go back to
Design
experiments
(e.g. A/B)
Line-in-the-sand
was crossed
Implement what
seems to work
Find correlations
between those
patterns & users
data
available
no data
Analyze the
behavior of the KPI
Line-in-the-sand
was NOT crossed
Discovery.
Talk to customers
Try again
Identify business objectives &
decision processes

what

Good metrics are:
• Actionable Accessible Auditable
• Metrics need to change a behavior - or need to hurt
• Use rates
• One Metric That Matters:
based on model and stage

Some terms to familiarise yourself with:
• Qualitative versus quantitative metrics Qualitative metrics are unstructured, anecdotal, revealing, and
hard to aggregate; quantitative metrics involve numbers and statistics, and provide hard numbers but
less insight.
• Vanity versus actionable metrics Vanity metrics might make you feel good, but they don’t change how
you act. Actionable metrics change your behavior by helping you pick a course of action.
• Exploratory versus reporting metrics Exploratory metrics are speculative and try to find unknown
insights to give you the upper hand, while reporting metrics keep you abreast of normal, managerial,
day-to-day operations.
• Leading versus lagging metrics Leading metrics give you a predictive understanding of the future; lag-ging
metrics explain the past. Leading metrics are better because you still have time to act on them.
• Correlated versus causal metrics If two metrics change together, they’re correlated, but if one metric
causes another metric to change, they’re causal. If you find a causal relationship between something
you want (like revenue) and something you can control (like which ad you show), then you can change
the future.

Typically your focus will go through the fives stages
of product/business maturity
Gate
Stage
EMPATHY I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE The users and features fuel growth organically
and artificially.
SCALE I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
The five stages

More terms to familiarise with:
Average Revenue Per User (ARPU): Revenues / Users
Churn: users abandoned (month)
Lifetime: 1 / churn
Lifetime Value (LTV): ARPU * Lifetime
Customer Acquisition Cost (CAC):
Total monthly acq costs / new monthly customers
(e.g. 3xCAC < LTV)

Even more terms to familiarise yourself with:
Invitation Rate = Invited / inviters
Acceptance Rate = Signed-up / Invited total
Viral Coefficient = Invitation Rate * Acceptance Rate
(e.g. Users * VC - churn > 1)

(Money from ad clicks)
Empathy
Stickiness
Virality
Revenue
Scale
E-commerce
Mobile
app
User-gen
content
SaaS Media
2-sided
market
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
CAC, shares,
reactivation
(Money from transactions)
Transaction,
CLV
Affiliates,
white-label
Engagement,
churn
Inherent
virality, CAC
(Money from active users)
Upselling,
CAC, CLV
API, magic #,
mktplace
Content,
spam
Invites,
sharing
Ads,
donations
Analytics,
user data
Inventory,
listings
SEM, sharing
Transactions,
commission
Other
verticals
Downloads,
churn, virality
WoM, app
ratings, CAC
CLV,
ARPDAU
Spinoffs,
publishers
Traffic, visits,
returns
Content
virality, SEM
CPE, affiliate
%, eyeballs
Syndication,
licenses
Lean Analytics
Important: drawing base-lines (in the sand)
A considerably more information-rich version is available at: bit.ly/BigLeanTable

What KPIs do you use at the moment?
What is the example of the impact of these in your business?

3. What matters?
Money-talks

TL;DR
1.Make more money than what you spend *

TL;DR
1. Make more money than what you spend *
2. The (dream of) Hockey-stick growth includes a long painful
period with resource stagnation
(you are working on high growth ventures right?)

- SCALE STAGE
8
THE STARTUP FUNDING SPECTRUM
NET CASH
POSITION
+
Closed
ALPHA
Public
BETA
1st Revenue
Redistributed for educational purposes with permission from Maria Dramalioti Taylor, Managing Partner, x.Million Capital Ventures

TL;DR
2. The (dream of) Hockey-stick growth includes a long painful period
with resource stagnation
3. Finance people want numbers, charts, proof; be it investors or
CFOs
4. Planning and playing with other people’s money involves
accountability. Be accountable for the right thing

The “Business Plan” Approach
(top-bottom)
• Do background / buy research
• Check macro - trends
• Make projections and forecasts
Initially to convince… and find
yourself accountable 12 months
later…
The Data Driven Approach
(bottom-up)
Deductively and quantitatively prove that:
• (Startups *)
…you are narrowing-in a valid Business
Model
• (Existing businesses / products )
…your key growth assumptions still
hold-true; optimize

What planning, forecasting and
optimization processes do you use at the moment?
What are your challenges with these at the
moment?

Hopefully by now we all agree that:
2. The (dream of) Hockey-stick growth includes a long painful
period with resource stagnation
3. Playing with other people’s money involves accountability. Be
accountable for the right thing
4.Data-driven planning & decisions (*) reduce risk

4. Perfect

A few common problems when
implementing a data-driven business
1. The asterisk in decisions was on purpose.
data-driven decisions need analytics AND decision processes
2. The 2nd asterisk in decisions. What’s hard about pivoting?
alignment, transparency, knowledge
3. Vanity metrics
… and politics. Someone will look ugly.
4. Resourcing
inability to explain / understand the importance
5. Culture (I know all) / Ethics (Morale in changing situations)
Change is tough

George @ Voulgaris .net
@gevou
fallingelephant.com visionmobile.com techtalkscentral.com

1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (19)

Similar to 1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris

Similar to 1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris (20)

More from prodactive

More from prodactive (17)

Recently uploaded

Recently uploaded (20)

1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris