Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)

Lean Analytics
for Intrapreneurs
Lean Startup Conference 2013
@acroll

When you’re a startup
your goal is to ﬁnd a sustainable,
repeatable business model.
When you’re a big company
your goal is to perpetuate one.

Lean Analytics:
Use data to build a better business faster.

Intrapreneur:
Someone working to produce
disruptive change in an organization
that has already found a sustainable,
repeatable business model.

(Before we get into Lean Analytics, 2 key lessons.)

Lesson one:
Companies die because they fail to
move to new business models.

$1

8”

5.25”

3.5”

Clay Christensen, The Innovator’s Dilemma

Time

eb

oo

k

$1000

N
ot

op
kt
De
s

ic
in
M

$100

ai

nf

ra

om

m

e

pu

te

r

$10

M

Cost per MB

14”

Technologies
outstrip what
the market
needs, driven
by feedback
from the
“best”
current
customer.

$1

8”

5.25”

$10

end
High er
stom
cu

end
Low
mer
usto
c

$100

$1000

Time

The new
market has
different criteria
for success,
which are
uninteresting to
incumbents.

$1

$10

$100

Storage
capacity

Portability

$1000

Time

Sometimes
this has
unintended
consequences

$1
Smaller disc size means less vibration
impact, leading to greater density,
increasing storage capacity
$10

$100

$1000

Time

Three kinds of innovation
Improve along
current metrics...

Switch to a new
value model

Change the business
model entirely

...or alter
the rate of
improvement

Sustain/core

Innovate/adjacent

Disrupt/transformative

(optimizing for more of the same)

(introduce nearby product,
market, or method)

(Fundamentally changing
the business model)

Lesson two:
The difference between a rogue agent
and a special operative is permission.

It’s not your job to prove you’re clever.
It’s your job to change outcomes.

A quick introduction
to Lean Analytics

Kevin Costner is a lousy entrepreneur.

Don’t sell what you can make. Make what you can sell.

The core of Lean
is iteration.

Unfortunately,
we’re all liars.

Everyone’s idea is
the best right?
No data, no
learning.

People love
this part!
(but that’s not always
a good thing)

This is where
things fall apart.

Analytics is the measurement of
movement towards your business
goals.

In a startup, the purpose of analytics is
to iterate to product/market ﬁt
before the money runs out.

In a big company,
analytics replaces opinion with fact.

It helps you innovate on product,
market, and method.

A good metric is:
Understandable

Comparative

If you’re busy
explaining the
data, you won’t
be busy acting
on it.

Comparison is
context.

A ratio or rate

The only way to
measure
change and roll
up the tension
between two
metrics (MPH)

Behavior
changing

If you’re busy
explaining the
data, you won’t
be busy acting
on it.

The
simplest
rule

If a metric won’t change how
you behave, it’s a

bad
metric.

h"p://www.ﬂickr.com/photos/circasassy/7858155676/

Metrics help you know yourself.
Customers that
buy >1x in 90d

1-15%
15-30%
>30%

Then you are
in this mode

Your customers
will buy from you

You are
just like

70%

Acquisition

Once

Hybrid

2-2.5

20%

per year

of retailers

Loyalty

>2.5

10%

per year

of retailers

of retailers

Focus on

Low acquisition
cost, high checkout

Increasing return
rates, market share

Loyalty, selection,
inventory size
(Thanks to Kevin Hillstrom for this.)

Qualitative

Quantitative

Unstructured, anecdotal,
revealing, hard to
aggregate, often too
positive & reassuring.

Numbers and stats.
Hard facts, less insight,
easier to analyze; often
sour and disappointing.

Warm and fuzzy.

Cold and hard.

Exploratory

Reporting

Speculative. Tries to ﬁnd
unexpected or
interesting insights.
Source of unfair
advantages.

Predictable. Keeps you
abreast of the normal,
day-to-day operations.
Can be managed by
exception.

Cool.

Necessary.

Rumsfeld on Analytics
we know

Are facts which may be wrong and
should be checked against data.

we don’t
know

Are questions we can answer by
reporting, which we should baseline
& automate.

we know

Are intuition which we should
quantify and teach to improve
effectiveness, efﬁciency.

we don’t
know

Are exploration which is where
unfair advantage and interesting
epiphanies live.

know

Things we
don’t
know

(Or rather, Avinash Kaushik channeling Rumsfeld)

Slicing and dicing data
Active users

5,000

Cohort:
Comparison of
similar groups
along a timeline.

0

Jan

(this is the April cohort)

Feb

Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)

Mar

☀
☁

Apr

May
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)

☀
☁
☀
☁

Multivariate
analysis
Changing several
things at once to
see which correlates
with a result.

Which of these two companies
is doing better?

March

April

May

Rev/customer

$5.00

$4.50

$4.33

$4.25

$4.50

1

2

3

4

5

January
How about
this one?

February

Cohort

Is this company
growing or stagnating?

January

$5

$3

$2

$1

$0.5

$6

$4

$2

$1

March

$7

$6

$5

April

$8

$7

$9

February

May

Cohort

2

3

4

5

January
Look at the
same data
in cohorts

1
$5

$3

$2

$1

$0.5

February

$6

$4

$2

$1

March

$7

$6

$5

April

$8

$7

May

$9

Averages

$7

$5

$3

$1

$0.5

The ability to do things in small batches
is behind much of the lean movement.

Short cycle time triggers the immune
system of big company budgeting.

(this is called foreshadowing.)

Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.

Explaining the
past.

Leading
Forward-looking.
Number today that
predicts tomorrow;
reports the news.
Example: pipeline.

Predicting the
future.

Some examples
A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one ﬁle in one folder on one device
(ChenLi Wang)
Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)

Which means it’s time to talk
about correlation.

10000
1000
100
10
1
Jan Feb

Mar

Apr

May

Ice cream consumption

Jun

Jul

Aug

Sept

Drownings

Oct

Nov

Dec

Correlated
Two variables that are
related (but may be
dependent on
something else.)

Ice cream &
drowning.

Causal
An independent variable
that directly impacts a
dependent one.

Summertime &
drowning.

A leading, causal metric
is a superpower.
h"p://www.ﬂickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/

Growth hacking, demystiﬁed.

Pick a metric
to change

Find
correlation

Test
causality

Optimize the
causal factor

http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/

Is social action a leading
indicator of donation?

http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/

Is mobile use?

Eric’s three engines of growth
Stickiness

Virality

Price

Approach

Keep people
coming back.

Make people
invite friends.

Spend money to
get customers.

Math that
matters

Get customers
faster than you
lose them.

How many they
tell, how fast they
tell them.

Customers are
worth more than
they cost.

Dave’s Pirate Metrics
AARRR

Acquisition
Activation
Retention
Revenue
Referral

How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...

Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...

Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...

Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...

Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...

The five stages

Stage
EMPATHY
STICKINESS

Gate
I’ve found a real, poorly-met need that a
reachable market faces.
I’ve figured out how to solve the problem in a
way they will keep using and pay for.

VIRALITY

I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.

REVENUE

The users and features fuel growth organically
and artificially.

SCALE

I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.

Empathy stage:
Localmind hacks Twitter
Needed to ﬁnd out if a core assumption—strangers answering
questions—was valid.
Ran Twitter experiment instead of writing code
Asked senders of geolocated Tweets from Times Square random
questions; counted response rate
Conclusion: high enough to proceed

Stickiness stage:
qidiq streamlines invites
Survey owner adds recipient to group

Survey owner adds recipient to group

Survey owner asks question

Recipient gets invite
Recipient installs mobile app
Recipient creates account, profile
Recipient can edit profile, etc.
Recipient reads survey question
Recipient responds to question
Recipient sees survey results

70-90% RESPONSE RATE

10-25% RESPONSE RATE

Survey owner asks question

Recipient reads survey question
Recipient responds to question
Recipient sees survey results
(Later, if needed…)
Recipient visits site; no password!
Recipient does password recovery
One-time link sent to email
Recipient creates password
Recipient can edit profile, etc.

Six business model archetypes
(Yours is probably a blend of these.)

E-commerce
SaaS (freemium?)
Mobile app (gaming)
Two sided marketplace
Media
User generated content

Customer Acquisition Cost
paid

direct

search

wom

(Which means eye
charts like these.)

Viral coefficient
Viral rate

inherent
virality

VISITOR
Freemium/trial offer
Invite Others

Upselling
rate

Upselling

Enrollment
Capacity Limit

User

Disengaged User

Free user
disengagement

Paid
conversion

Engaged User

Tiering

Paying Customer
Support data

Reactivation
rate

Freemium
churn

Reactivate

Trial Over

Disengaged

Dissatisfied

Trial abandonment
rate

Resolution

Cancel

Cancel

Reactivate
Account Cancelled

Billing Info Exp.

Paid Churn Rate
FORMER USERS
User Lifetime Value

FORMER CUSTOMERS
Customer Lifetime Value

Model + Stage = One Metric That Matters.
The business you’re in

The stage you’re at

E-Com

SaaS

Mobile

2-Sided

Media

Empathy
Stickiness
Virality
Revenue
Scale

One Metric
That Matters.

UCG

In a startup, focus is hard to achieve.

Having only one metric
addresses this problem.

Moz cuts down on metrics
SaaS-based SEO toolkit in the scale stage. Focused on net adds.

Net adds up:
Net adds ﬂat:

Net adds down:

Was a marketing campaign successful?
Were customer complaints lowered?
Was a product upgrade valuable?
Can we acquire more valuable customers?
What product features can increase engagement?
Can we improve customer support?
Are the new customers not the right segment?
Did a marketing campaign fail?
Did a product upgrade fail somehow?
Is customer support falling apart?

Metrics are like squeeze toys.

http://www.ﬂickr.com/photos/connortarter/4791605202/

Ecommerce

Empathy
Stickiness

Virality

2-sided
market

Scale

Mobile
app

User-gen
content

Media

Interviews; qualitative results; quantitative scoring; surveys

Loyalty,
conversion

Inventory,
listings

CAC, shares,
SEM, sharing
reactivation
(Money from transactions)

Revenue

SaaS

Engagement, Downloads,
churn
churn, virality

Content,
spam

Traffic, visits,
returns

Inherent
virality, CAC

Invites,
sharing

Content
virality, SEM

WoM, app
ratings, CAC

(Money from active users)

Transaction,
CLV

Transactions,
commission

Upselling,
CAC, CLV

CLV,
ARPDAU

Affiliates,
white-label

Other
verticals

API, magic #, Spinoffs,
mktplace
publishers

(Money from ad clicks)
Ads,
donations

CPE, affiliate
%, eyeballs

Analytics,
user data

Syndication,
licenses

A company loses a quarter of its
customers every year.
Is this good or bad?

Not knowing what normal is
makes you do stupid things.

Listening to what normal is
makes you ignore disruptive things.

Baseline:
5-7% growth a week
• Are there enough people who really care

enough to sustain a 5% growth rate?
• Don’t strive for a 5% growth at the expense
of really understanding your customers
and building a meaningful solution
• Once you’re a pre-revenue startup at or
near product/market ﬁt, you should have
5% growth of active users each week
• Once you’re generating revenues, they
should grow at 5% a week

“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doing exceptionally well. If you
can only manage 1%, it's a sign
you haven't yet ﬁgured out
what you're doing.” At revenue
stage, measure growth in
revenue. Before that, measure
growth in active users.
Paul Graham, Y Combinator

Baseline:
10% visitor engagement/day

30%

of users/month use web or mobile app

10%

of users/day use web or mobile app

1%

of users/day use it concurrently
Fred Wilson’s social ratios

Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.

Baseline:
CLV calculation

25%

5%

2%

monthly churn

monthly churn

monthly churn

100/25=4

100/5=20

100/2=50

The average
customer lasts
4 months

The average
customer lasts
20 months

The average
customer lasts
50 months

Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time to ﬁnd
out whether your churn and revenue projections
are right
• Cashﬂow: You’re basically “loaning” the
customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a
CAC of only a third of your CLV will forces you
to verify costs sooner.

20 mo. CL
$30/month per
customer
$600 CLV
1/3 spend
$200 CAC
Now segment
those users!

Putting this to work:
The Lean Analytics Cycle

Pick a KPI

Success!
Pivot or
give up

Draw a new line
Try again

Did we move the
needle?

Measure
the results

Draw a line

Find a potential
improvement

Without data:
make a good
guess

With data:
ﬁnd a
commonality

Design a test
Hypothesis
Make changes
in production

Do AirBnB hosts
get more business
if their property is
professionally
photographed?

Gut instinct (hypothesis)
Professional photography helps AirBnB’s business

Candidate solution (MVP)
20 ﬁeld photographers posing as employees

Measure the results
Compare photographed listings to a control group

Make a decision
Launch photography as a new feature for all hosts

5,000 shoots per month
by February 2012

SRSLY?
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business

Without data: make a
good guess

With data:
ﬁnd a commonality

“Gee, those
houses that do
well look really
nice.”

“Computer: What
do all the
highly rented
houses have in
common?”

Maybe it’s the
camera.

Camera model.

Landing page design

A/B testing

URL shortening

Publisher analytics

Cohort analysis

General analytics

Funnel analytics

SaaS analytics

User analytics

Spying on users

Inﬂuencer Marketing

Gaming analytics

User segmentation

User interaction

Customer satisfaction

KPI dashboards

Agenda
An introduction to
Lean Analytics (30m)

The challenges of
being big (15m)

When you have
support (30m)

The Lean Analytics
framework (30m)

A dose of
pragmatism (15m)

Metrics for innovation
portfolios (15m)

Break (15m)

Some non-tech
examples (10m)

Tools of the trade
(15m)

When it’s you against
the world (20m)
Break (15m)

It’s diﬀerent when you’re big.

It is way too easy to
mix these up.

Target
market

B2B

B2C

Early stage

Big/incumbent

Company size/age

Less WoM
More formal decisions

Intrapreneurs

Business model vs.
company stage

Slower cycle time
More legacy constraints

If big ﬁrms
can’t innovate,
it’s this guy’s
fault.

When product and market are known,
companies compete on how they do
things.

To get the incremental cost to zero,
companies competed on scale.

(Literally, an economy of scale)

Scale comes from process, IP, org
chart, capitalization.
All of these assume the future will be
like the past, only more so.

If a startup is an organization designed
to search for a sustainable, repeatable
business model, then an established
company is an organization designed
to perpetuate one.

Technology has radically changed the
incremental cost of businesses.

Software is eating the world.

http://www.ﬂickr.com/photos/ebolasmallpox/3733059220/

An economic order quantity
of one.
Crafted

Massproduced

Automated

Digital

Quantity

Few

Many

Some

One

Cost

High

Low

Medium

Free

Lead time

Small

Large

Medium

None

Self-service

Medium

None

Some

Lots

Customization

High

None

Some

Lots

This is why
software is
eating the
world.

• Cloud computing
• Social media
• 3D printing
• Per-customer
analysis
• Mobile tracking
• Etc...

Sustainable competitive advantage allows for
inertia and power to build up along the lines of
an existing business model, which will soon die.
Instead, seek transient competitive
advantage.

Rita Gunther McGrath, The End of Competitive Advantage

Scale is now a liability. Compete on
cycle time.

Optimizing the probable means
discounting the possible.

This isn’t about a lack of resources.

http://www.flickr.com/photos/maladjusted/5207565912

Blockbuster had a lot going for it.

Plenty of inventory, of course.
But that matters less than...

...market intelligence,
customers, existing
payment approval,
and customer
history.

The problem was framing:
Blockbuster thought it was in the video
store management business. Netﬂix
realized it was in the entertainment
delivery business.

OPTIMIZATION
OF CURRENT
METRICS

LOCAL
MAXIMUM

YOU ARE
HERE

INNOVATION
WITH NEW
RULES

YOU ARE
HERE

GLOBAL
MAXIMUM

YOU ARE
HERE

SHORT-TERM INVESTORS
HATE GOING DOWNHILL

First mover advantage happens
long before the market emerges.
• $1B invested in Nook
• $475M operating loss
in April 2013
• CEO gone

Constraints slow things down

vs.

Everything to lose:
Why big companies need innovation.

F500 Life
Expectancy
(http://csinvesting.org/2012/01/06/fortune-500-extinction/)

75

15

years

years

1950
Growth by entering
a new business

...

2010

95

99

% fail

% fail

Corporate
Strategy Board

Clay
Christensen

In other words, if your job is change you
have your work cut out for you.

2011 MIT study of 179 large publicly traded ﬁrms

Companies that use data-driven
analytics instead of intuition have
5%-6% higher productivity and
proﬁts than competitors.
Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven
Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).

Many models for enterprise innovation
Core

Adjacent

Transformative

Do the same
thing better.

Nearby product,
market, or method.

Start something
entirely new.

Regional
optimizations.

Innovation, go-tomarket strategies.

Reinvent the
business model.

• Customer development
• Test similar cases
• Parallel deployment
• Analytics & cycle time

• Fail fast
• Skunkworks/R&D
• Focus on the search
• Ignore the current
model & margins

• Get there faster
• Smaller batches
• Solution, then testing
• Increased accountability

Another way to look at it
Core

Adjacent

Transformative

Know the problem
(customers tell you it)
Know the solution
(customers/regulations/
norms dictate it.)

Know the problem
(market analysis)
Don’t know the solution
(non-obvious innovation
confers competitive
advantage.)

Don’t know the problem
(just an emerging need/
change)
Don’t know the solution.

Waterfall:
Execution
matters

Agile/scrum:
Iteration
matters

Lean Startup:
Discovery
matters

The Three Horizons
Core

Adjacent

Horizon 1 improves the
Horizon 2 extends the
current business operations business into new products,
in the next 12 months.
markets, or methods in the
next 3 years.
Those core businesses most
readily identified with the company
name and those that provide the
greatest profits and cash flow.
Maximize remaining value.

Emerging opportunities, including
rising entrepreneurial ventures
likely to generate substantial
profits in the future but that could
require considerable investment.

Transformative
Horizon 3 changes the
industry you’re in and your
value network in the next 6
years.
Ideas for profitable growth down the
road—for instance, small ventures
such as research projects, pilot
programs, or minority stakes in new
businesses.

http://www.mckinsey.com/insights/strategy/enduring_ideas_the_three_horizons_of_growth

Lean applies. Startup may not.
Core

Adjacent

Transformative

Lean
startup

Lean methodologies.

Method
(new “how”)

3 kinds of
innovation

Product
(new “what”)
Market
(new “who”)

Another view

Core

Method
(new “how”)
Market
diversiﬁcation

Distribution
innovation

Pure
startup

Adjacent
Disruptive
Inventive

Market
(new “who”)

Channel
expansion

Product
(new “what”)

A three-maxima model
of enterprise innovation
Business
optimization
(ﬁve mores)
Current
state

Business
model
innovation

Product,
market,
method
innovation
You can convince
executives of this
because some of it
is familiar.

This terriﬁes them
because it eats the
current business.

Improvement

Adjacency

Remodeling

Do the same,
only better.

Explore what’s
nearby quickly

Try out new
business models

Lean approaches apply, but the metrics vary widely.

Sustain/
core

Innovate/
adjacent

Disrupt/
transformative

More things

To more people

Sustaining
innovation
is about
more of
the same.
(says Sergio Zyman)

Inventory increase
Gifting, wish lists
Highly viral offering
Low incremental order costs

For more money

Maximum shopping cart
Price skimming/tiering

More often

Loyal customer base that returns
Demand prediction, notiﬁcation

More eﬃciently

Supply chain optimization
Per-transaction cost reduction

Software, experimentation, and
iterative cycles of learning help you
get to the local maximum better and
faster. That’s a good thing.
But it’s not the only thing.

Adjacent innovation is about changing
one part of the model in a way that
alters the value network.

Adjacent product
to the same
market in the
same way

Selling the same product to an
adjacent market in the same
way.
Of P&G’s 38 brands, only 19 were sold in Asia as of 2011
Market expansion is seldom selling the same thing to new people. In
Asia, P&G needed to
Align pricing with novelty (prestige, mass-tige, over-the-counter)
Change consumer expectations (moving from dilutes to
concentrates)
Adjust positioning and ingredients such as white fungus, ginseng,
and the parasitic cordyceps

Selling the same product to the same in
The biggest innovation
logistics of the 20th century.
market in a new way.

http://www.flickr.com/photos/photohome_uk/1494590209

Selling the same product to the
same market in a new way.

(At this point, observant Intrapreneurs
should be asking, should P&G be in
the house cleaning business?
And that would be transformative.)

Transformative innovation is about
taking a leap, changing more than one
dimension simultaneously in search of
a new business model.

If sustaining, incremental innovation
produces linear growth, then
disruptive, transformative innovation
produces exponential growth.

Transformative isolation:
Skunkworks

I lied. Everyone is a tech company.

Cost of experiments: down.
http://www.ﬂickr.com/photos/puuikibeach/4789015423

Cost of attention: way up.
http://www.ﬂickr.com/photos/elcapitanbsc/3936927326

Let’s pick on restaurants
for a while.

Empathy: ﬁnd the need
Before opening, the owner ﬁrst learns about the diners in her
area, their desires, what foods aren’t available, and trends in
eating.

Key metrics: Popular items;
frequent questions; before/after
dining patterns.

Reference: Emerging need.

Stickiness: conﬁrm the need is met.
She develops a menu and tests it out with consumers,
making frequent adjustments until tables are full and patrons
return regularly. She’s giving things away, asking diners what
they think. Variance and uncertain inventory make costs high.
Key metrics: Customer loyalty;
recommendations; referrals;
endorsements; inventory turnover.

Reference: Business idea.

Virality: will it spread?
She starts loyalty programs to bring frequent diners back, or
to encourage people to share with their friends. She engages
on Yelp and Foursquare.

Key metrics: Customer loyalty;
recommendations; referrals;
endorsements.

Reference: Business positioning

Revenue: prove the business viability
With virality kicked off, she works on margins—fewer free
meals, tighter controls on costs, more standardization. She
focuses on the price of acquiring new customers.

Key metrics: Acquisition cost,
revenue per cover, capacity,
turnover.

Reference: Business model.

Scale: prove it’s a market
Knowing she can run a proﬁtable business, she funnels
revenues into marketing and promotion. She reaches out to
food reviewers, travel magazines, and radio stations. She
launches a second restaurant, or a franchise.
Key metrics: Franchise health;
repeatability; problems escalated;
variance; franchise revenues.

Reference: Business plan.

A line in the sand

30%
Labor costs
Gross revenue

= 24%
20%

Too costly?
Just right
Understaffed?

A leading indicator
(Varies by
restaurant.
McDonalds
≠ Fat Duck.)

50 reservations
at 5PM
http://www.ﬂickr.com/photos/mysticcountry/3567440970

250 covers
that night
http://www.ﬂickr.com/photos/avlxyz/4889656453

Restaurant MVP

http://www.ﬂickr.com/photos/southbeachcars/6892880699

Is tip amount a leading indicator of longterm revenue?

Why does every table get the same
menu?

Is purple ink better?
http://tippingresearch.com/uploads/managing_tips.pdf

Stalking
customers
is pretty
easy.

http://targetmycustomers.appspot.com

http://tippingresearch.com/uploads/managing_tips.pdf

When it’s you vs. the world.
(A bagful of tricks from agitators in companies of all sizes.)

The skills you need make you a pariah.
Successful innovators share certain attributes.
Bad listener: Willfully ignore feedback from your best customers.
Cannibal: If successful, destroying existing revenue streams.
Job killer: Automation & lower margins are your favorite tools.
Security risk: Advocate of transparency, open data, communities.
Narcissist: Worry constantly about how you’ll get attention.
Slum lord: Sell to those with less money, deviants, and weirdos.

New

Know what
kind of
innovation
you’re
Market
after.

Current
Based on H. Igor Ansoﬀ’s matrix

Startup:
New products for new
!)
markets. News rules,
es
cc units,
business
su
at
organizational
re
g
nd
structure. Innovation.
(a

Market development:
Sell existing products
to new markets,
segments, uses.
Export & license.
lf
a

ll
a

Penetrate: itic
ol
Increase revenues,
fp
o
market share, product
sk
ri
ed
quality, brand
as
re
differentiation.
nc
I
Marketing.
Current

ut
o

Product
development:
Invent new products
for your market. R&D,
enhancements.
Acquisition.

Product

New

Frame it like a study
Product creation is almost
accidental.
Unlike a VC or startup, when
the initiative fails the
organization still learns.

http://www.flickr.com/photos/creative_tools/8544475139

When in doubt, collect data
From tackling the FTA rate to
visualizing the criminal justice
supply chain.

Use data to create a taste for
data
Sitting on Billions of rows of
transactional data
David Boyle ran 1M online surveys
Once the value was obvious to
management, got license to dig.

Don’t just collect data,
chase it.

Understand hidden
constraints
That pencil story is a myth.
Graphite is conductive and
explosive. The Minimum
Viable Product is Viable for a
reason.

Know what has to
be built in-house
SAP integration
Employment law

Think subversively.

http://www.ﬂickr.com/photos/bootbearwdc/1243690099/

Everything’s an excuse to experiment

Run it as a consulting business ﬁrst.
(Just don’t get addicted to it. Your goal is to
learn and overcome integration challenges and
ﬁnd the 20% of features that 80% of the market
will pay for.)

Convince your boss she asked for this
Success!

Pick a KPI
Pivot or
give up

Draw a new line
Try again

Did we move
the needle?

Measure
the results

Draw a line
in the sand
Find a
potential
improvement
Without
data: make a
good guess

With data:
ﬁnd a
commonality

Design a test
Make changes
in production

Hypothesis

Slaughter a sacred cow:
Prove a long-held assumption is
wrong and you’ve got people’s
attention.
Know what you’ll do with it ahead of
time.

Tesla

http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg

Twitter’s 140-character
limit isn’t arbitrary. It’s
constrained by the size
of SMS (160 characters)
and username (20
characters.)

http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/
sms_screen_twitter_activity_stream_270x405.png

Figure out how to translate it back to a
simple model that ﬁts the company’s
existing value model.
If your company dies, this is why.

Intrapreneurs often have to use proxies
Stage

Startup metrics

Intrapreneur metrics

Empathy

Customers interviewed (needs &
solutions), assumptions quantified,
TAM, monetization possibility

Non-customers interviewed;
assumptions quantified, constraints
identified, TAM, disruption potential

Churn, engagement

Support tickets, integration time, call
center data, delays

Viral coefficient, viral cycle time

Net Promoter Score, referrals, case
study willingness

Revenue

Attention, engagement

Billable activity; signed LOIs; pilot
programs; after-development
profitability

Scale

Automation

Contribution, training costs, licensing

Stickiness
Virality

When you have support.
(What companies like P&G, Cognizant, GE, and Motorola do with a
formal innovation program.)

Do you really have permission?
What resources do you have?
Staff, budget, unfettered access to customers?
What scope of change can you make?
Pricing, product, channel, branding?

Step one: Develop a portfolio approach.

Innovation portfolios at big companies

Return

Investment

Core

Adjacent

70%

20%

10%

20%

Transformative
10%

70%

1.

Organizations’ structures emerge as a way
to optimize the current business model.

2.

Most innovations will come not from
product or market, but from method—
business model innovation.

3.

Innovation groups must exercise
organizational amnesia at the outset.

Step one: Frame your problems.

Maybe not
so crazy.

http://www.ﬂickr.com/photos/mizrak/4592706544

Step two: Deﬁne your gates and ﬁlters.
These may lead to myopia.
They are also your unfair advantages.

Step three: Secure funding, resources, and
executive backing.

Step four: Generate new ideas.

Find non-obvious adjacencies
POWER GRID

WHICH
FEEDS A
NEEDS
LIGHT AN

BULB

ELECTRICAL
GENERATOR

HAS A
TURBINE
LIKE A

TURNED
AROUND
BECOMES A

PLANE ENGINE

TRAIN ENGINE

REQUIRES

SPINS &
VIBRATES
LIKE AN

AND
LOOKS
LIKE A

SOFTWARE TO
CUT DOWN
TREES BETTER

MRI MACHINE

WIND TURBINE

Build an ecosystem
Canada’s largest directory
publishing and local
marketing services company
1.5M listings from 420K
SMB & national customers
Revenues >$1.2B
2,500 employees
Created third-party listing API
Took 8-10 mo (2009-10)
to get approval

API payoff happened 2y later

KPI evolution

Yahoo replaced Canadian
digital properties search
with the YellowAPI

Soft: Signups,
SDK, downloads

Improved SEO, Comscore

App usage,
deals signed

Functional prototype in
hours, testing in days, and
launching in weeks.
Faster time to partnerships
Budgets tripled in 2013

API calls
generated
API-generated
revenue

Five common models for
transformative innovation
Acquisition
Collaboration
Isolation

Buy promising startups
Crowdsource, work with
universities, suppliers, etc.
Create a separate group
with different conditions

Incubation

Internal startup ecosystem;
LoB are “investors”

Integration

The LoB does innovation
internally

Step ﬁve: Test by doing (experimentation
beats projection.)

Focus on
the model,
not the
plan

Demand

Amt

Growth Wk 1 Wk 2 Wk 3

People per day on sidewalk

200

4%

204

Percent that buy a glass

10%

5%

15% 20% 25%

Daily customers

20

31

Revenue
Price per cup

Proﬁt per cup
Daily proﬁt

43

225
56

$156 $216 $281
$5

$5

Cost of Goods
Cost per cup

216

$5

$5

30.6 41.5 52.8
$1

-2%

.98

.96

.94

$4

4.02 4.04 4.06

$80

125

175

228

A business plan is just what happens
when you drag the business model to
the right.

Designing an experiment
Problem, solution, and result hypothesis
Test strategy (PoC, survey, interviews, kickstarter, prototype, A/B, etc.)
Cohort & segment to be tested
Metric or assumption being tested
Timebox or total for test
Action you’ll take if you pass or fail

Step six: Know what happens afterwards

Qualcomm’s innovation model:
What was missing
Hypothesis

Unclear what
happened to
founders
Needed a
middle PoC
decision
Sustainability,
not feasibility
http://blogs.berkeley.edu/2013/01/28/
designing-a-corporateentrepreneurship-program-aqualcomm-case-study-part-1-of-2/

Experiment
Boot
camp

Idea generation
and selection
Existing models

Implement
Idea
advancement

POC
Biz
sustainability

Tech
feasibility
POC

Ideas

New
models
Open
innovation

Strategic
value

decision
Boot
camp
decision

End user/partner
desirability

Option
value

Implement
decision

Actions

Exit
value

Company crowd storm Small team designs & Company extracts value
conducts experiments

The Lean Analytics lifecycle of an Intrapreneur
Get buy-in

Political fallout

Find problems; don’t test demand.
Skip the business case, do analytics

Entitled, aggrieved
customers

Stickiness

Know your real minimum based on
expectations, regulations

Hidden “must haves”,
feature creep

Virality

Build inherent virality in from the
start; attention is the new currency

Luddites who don’t
understand sharing

Revenue

Consider the ecosystem, channels,
and established agreements

Channel conﬂict,
resistance, contracts

Hand the baton to others gracefully

Hating what happens
to your baby

Beforehand
Empathy

Scale

Metrics for innovation portfolios.

Core metrics
Business plan.

Metrics that matter

Assume it will improve.

• Return on investment
• Total cost of ownership
• Improvement in KPI
• Total served market

Product, market, and method will
remain the same

Examples: Redesigning
packaging; pricing
adjustment

Adjacent metrics
Business model.

Metrics that matter

Assume it will fail.

• Virality & word of mouth
• Early adopter stickiness
• Regulation
• Total addressable market

Your ultimate use case won’t be
what you think it is today.

Example: Mr. Clean
Magic Eraser

Transformative metrics
Business idea.
Assume it will fail.
You hope it will have the
consequences you want but
aren’t sure how.

Example: Headcam
recordings of all ofﬁcers

Metrics that matter
• People I’ve talked to
• Prototype creation speed
• Assumptions validated
• Problems uncovered
• Technical feasibility
• Hidden constraints

Key points to clarify in an
innovation program
Hypothesis

Experimentation

• Articulate problems
• Frame known advantages
• Define the right filters
• Many idea sources
• Confirm funding (money,

• Prioritize riskiest

•

•

•

people, customer access)
Agree on analytical
framework
Balance market, product,
& method adjacencies

•
•

assumptions
Time-box assessment
stages
Test technology, demand,
and business feasibility
MVP, prototype, pilot, or
science as appropriate for
type of innovation

Implementation

• Temporary incubator
• Find a home or building
•
•
•

one
Keep innovators involved
Merge metrics with
existing business KPIs
Synchronize innovation
cycles with enterprise
cycles (budget, etc.)

Portfolio metrics; Gates and KPIs for each stage; mix of core, adjacent, and
disruptive innovation.

Hypothesis

Goals, constraints, context

Sourcing
Top-down
Bottom-up

Sourcing
Filtering
Biases
Strengths
Focus
Mandate

Outside-in

ost e
l alm as th
wil s w
this . thi
at
re a o
.
e th ange. here a d t
ot ch
N y
dt
nee
n
inl eck a ions I
erta P d
c
rat it.
MV of alte e to
ak
nch
m
bu

Boot camp

POC

Core
Optimization

Crosspollinate
to current
managers

Adjacent
De-risking
Best solution

Test/
validate
w/current
customers

Disruptive
Reframing
R&D

M&A

Grow
as a
distinct
business

Evaluation of the innovation program itself

Implementation
Socializing

Integration
Adoption by
existing line of
business.

Independence
Creation of a new
line of business.

Traction graphs
Your business model
The stage you’re at

... change often if
you’re doing it right.

Your one metric
So how do you track
that over time?

Traction graphs

Jan

Feb

Signups
per day
This axis changes for
each metric

Mar

Conversion
rate

Apr

Churn
rate

May

Jun

Viral
coefﬁcient

Traction graphs
0%

Jan

Signups
per day

Feb

Mar

Conversion
rate

Apr

Churn
rate

May

Jun

Viral
coefﬁcient

Use vanity to get to
meaningful metrics
Your goal is to produce
outcomes
If the outcomes require
action, and vanity motivates
actors, use it
But show how the vanity
metric is a leading indicator of
the real one

Web traﬃc

Activation

Cart
Size

Conversion
x
rate

Revenue

The three threes
Three
assumptions

What big bets are you making?
• “People will answer questions”
• “Organizers are frustrated with how to run conferences”
• “We'll make money from parents”
• “Amazon is reliable enough for our users.”

Three actions
to take

What are you doing to make these assumptions happen (or
identify they’re wrong and change course?)
• Product enhancements
• Marketing strategies

Three experiments
to run

• Feature tests
• Continuous deployment
• A/B testing
• Customer survey

The three threes
Three
assumptions

Monthly

Board, investors,
founders

Strategy

Three actions
to take

Weekly

Executive team

Tactics

Three experiments
to run

Daily

Employees

Execution

The three threes
Many people will
answer questions

Three
assumptions

Three actions
to take

Three experiments
to run

Get more
people

Change
the UI

Increase
answer %

Test
timings

Test better
questions

Questions
from peers

The problem-solution canvas
The Goal is to Learn
CURRENT STATUS

• List key metrics you’re

LAST WEEK’S LESSONS LEARNED
AND ACCOMPLISHMENTS)

• What did you learn last week?
tracking, where they’re at, and
• What was accomplished?
compare with last few weeks
• On track: YES / NO?
How are things trending?
•

The problem-solution canvas
Problem #1 (put name here)
HYPOTHESIZED SOLUTIONS
• List possible solutions that you’ll start

working on next week. Rank them.
• Why do you believe each solution will
help you solve or complete solve the
problem?

METRICS / PROOF + GOALS
• Metrics you’ll use to measure whether

or not the solutions are doing what you
hoped (solving the problem)
• List proof (qualitative) you’ll use as well
• Deﬁne goals for the metric

Problem #2 (put name here)
HYPOTHESIZED SOLUTIONS
• List possible solutions that you’ll start

working on next week. Rank them.

METRICS / PROOF + GOALS
• Metrics you’ll use to measure whether

or not the solutions are doing what you
hoped (solving the problem)

Use proxy data. Just be careful.

“A subjective degree of
belief should rationally
change to account for
evidence.”
(AKA Bayes’ Theorem.)

Key points
Intrapreneurship is about adjacent or transformative innovation
Sustaining innovation focuses on the Five Mores, within the
current product, market, method, and business model.
Adjacent innovation may come from a new product, market, or
method, but the same business model
Disruptive innovation has different customers, KPIs, and models
The difference between a rogue agent and a special operative is
permission
Portfolios need framing, sourcing, ﬁlters, metrics, and socializing
Balancing isolation and integration, R&D and M&A is contentious

“The most important ﬁgures that one
needs for management are unknown
or unknowable, but successful
management must nevertheless take
account of them.”
Lloyd S. Nelson

Pic by Twodolla on Flickr. http://www.ﬂickr.com/photos/twodolla/3168857844

ARCHIMEDES
HAD TAKEN
BATHS BEFORE.

Once, a leader convinced others
in the absence of data.

Now, a leader knows
what questions to ask.

Ben Yoskovitz
byosko@gmail.com
@byosko

Alistair Croll
acroll@gmail.com
@acroll

Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (6)

Similar to Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)

Similar to Lean Analytics for Intrapreneurs (Lean Startup Conf 2013) (20)

More from Lean Analytics

More from Lean Analytics (13)

Recently uploaded

Recently uploaded (20)

Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)