High Tempo Testing

•

26 likes•70,579 views

High Tempo Testing is about setting a weekly testing goal and finding a way to hit it. This requires a process, which is shared in these slides. The slides show how the right process can help you overcome the challenges of high tempo testing and hit your growth goals.

Sean Ellis @seanellis
High Tempo Testing
Moving Beyond Hustle to Building
a Predictable Growth Machine

Sean Ellis @seanellis
Early Growth: Twitter Focused
Feedback: “I find growth articles
on Twitter”
Followers: Easy to find people that
tweet growth articles
Content: Long tail of content ideal
for Twitter

Sean Ellis @seanellis
Goal: Keep Conversation Going

Sean Ellis @seanellis
Viability Achieved by Month 3

Sean Ellis @seanellis
Goals Not Met After First Year
3 Flat Months!

Sean Ellis @seanellis
“Inputs are what you
actually control.”
Andrew Chen

Sean Ellis @seanellis
Problem: Limited Testing
Had specific growth goals, but just said
“test more”.
Team was too busy for much testing...
Building product
Executing Twitter Playbook
Engaging with community

Sean Ellis @seanellis
Solution: High Tempo Testing
Launch 3 Tests Per Week
(Clear “input” goal)

Sean Ellis @seanellis
Result...
Started high
tempo testing

Sean Ellis @seanellis
Hustle › Growth Process

Sean Ellis @seanellis
High Tempo Testing Process
Build
Knowledge
Base
Unbridled
Ideation
Prioritize
Backlog
High
Tempo
Testing

Sean Ellis @seanellis
Challenges of High Tempo Testing
1. Enough ideas: Ideas are fuel for tempo
2. Prioritizing ideas: Finding best ideas to test
3. Implementing tests: Getting the resources
4. Accurate testing: Instrumenting & analyzing
5. Avoiding repeats: Not running same tests

Sean Ellis @seanellis
Challenge 1: Enough Ideas
Unbridled ideation from full team
Ideas across all vectors
Individual & Team Accountability
Acqusition
Activation
Retention
Revenue
Referral

Sean Ellis @seanellis
Challenge 2: Prioritizing Ideas
Formalize ideas in “Experiment Doc”
Include research, hypothesis, target lever...
Score on impact, confidence & ease

Sean Ellis @seanellis
Challenge 3: Implementing Ideas
Weekly growth meeting to plan/assign current
week’s tests. Accountability for previous week.
Recommend Autonomous growth team with
dedicated development, design, analysis...
Empower less technical team members w tools

Sean Ellis @seanellis
Challenge 4: Accurate Testing
Read
Peep Laja
(Next presenter)

Sean Ellis @seanellis
Challenge 5: Avoiding Repeats
Capture all tests in Knowledge Base
Ensure knowledge base shared, clear & searchable
Worked Failed Inconclusive

Sean Ellis @seanellis
High Tempo Test Requirements
Process People System
Build
Knowledge
Base
Unbridled
Ideation
Prioritize
Backlog
High
Tempo
Testing

Sean Ellis @seanellis
Winning Test Example via Process
Moved email collector
from bottom to top.
Result: 700% increase
in collected emails
Impact prediction was
4/10

Sean Ellis @seanellis
High Tempo Testing Advantages
1. Predictable Growth
2. Scaleable Growth
3. Easier Hiring
4. Address Bottlenecks

Sean Ellis @seanellis
Thank you!
@seanellis

What's hot

SaaStock 2018 Dublin. Everyone wants more sales. In this presentation, David Skok unlocks the secrets to how he has been able to work with company after company and help them discover breakthroughs that drastically improve their conversion rates. The secret behind it all is become a master at understanding how your buyer thinks as they go through their purchasing journey, and as they experience your marketing and sales.

Get inside your Buyers Head - Improve Funnel Conversion Rates

David Skok

Why Startups Suck at Marketing

Rand Fishkin

Content Marketing Lessons From 10 Years and 41,525,103 Website Visits

Hiten Shah

Building a sales & marketing machine

David Skok

eCommerce Internal Linking - Into the Spider-Verse (BrightonSEO edition)

Kristina Azarenko

Playbook on how to build a B2B sales machine aka how to build a "$100 mln business". Delivered this talk with a few organizations including R3 Corda Seed+Series-A Startups, B Work Bali startup community. A few topics covered included: ⮞ Christoph Janz’ / Boris Wertz animal framework on how your hunting strategies differ for customers at different ARPA level ⮞ the anatomy of #growth engines ⮞ Deep dive into the #B2B outbound playbook for seed level #startups with limited budgets.

Slash | How to build a B2B sales machine

Andries De Vos

128 High Converting Growth Hacks - the most epic growth hacking list

Helvijs Smoteks

Dan Olsen, The Lean Product Playbook , @danolsen Room: C260 Everyone working on a new product is trying to achieve the same goal: product-market fit. Although product-market fit is one of the most important Lean Startup concepts, it’s also the least well defined. Dan Olsen shares the top advice from his book The Lean Product Playbook, including the Product-Market Fit Pyramid: an actionable model that breaks product-market fit down into 5 key elements. Dan also explains the Lean Product Process, a 6-step methodology with practical guidance on how to achieve product-market fit, illustrated with a real-world case study.

A Playbook for Achieving Product-Market Fit

Lean Startup Co.

Mobile Growth: Best Strategies, Tools and Tactics

Adrien Montcoudiol

Content writers: will AI take your job?

KatieThompson74137

Building a Sales and Marketing machine for a B2B software company involves many functions working together. This slide deck explores the process of funnel design that is highly buyer centric. It looks at the Buyer journey, and how to successfully construct a buying process that they buyer will enjoy going through, which is very different from the typical sale process that is designed from the vendors standpoint, and fails because the buyer is not motivated to go through the steps.

The Art & Science of Growth Hacking

David Skok

If you’re being methodical about growth, analytics matters. For startups, analytics is about measuring the right metric, in the right way, to produce the change the business needs most at that point in time. That’s harder than it sounds: you need a solid understanding of your business model; an awareness of what’s most at risk; and a clear idea of where to draw the line between success and failure. Metrics measure not only the health of your business, but also your journey to product/market fit; the value of your company; and the reliability of your underlying infrastructure. Join Lean Analytics co-author Alistair Croll for an all-day, in-depth look at analytics, measurement, and working with data. We’ll cover: The five stages of growth every company goes through, and how they guide your choice of metrics Six business-model archetypes and their unique measurement challenges What “good enough” looks like for fundamental metrics How to think about cohorts, segments, percentiles, and histograms Measuring and aggregating infrastructure KPIs such as latency and availability Using the Lean Analytics cycle to improve through experimentation This workshop is relevant for people working in standalone startups and for corporate entrepreneurs. It will combine presentations, case studies, and interactive discussion of the audience’s specific measurement challenges. Attendees need not be technical but should come armed with a basic understanding of web analytics, business metrics, and their current business model, plus a willingness to share with one another.

Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...

Lean Startup Co.

JTBD for Marketering - Nopadon.pdf

Business of Software Conference

From Zero to a Million Users - Dropbox and Xobni lessons learned

Adam Smith

Content Marketing in 15 Minutes a Day for Restaurants

Ann Handley

There are seven key stages in a startup’s evolution from $0m to $50m in revenue. Understanding where you are in that evolution, and how to act at each stage is critical for success, as what is appropriate at one stage is not appropriate at another stage. David will lay out the roadmap, and detail the keys to success at each stage. The talk is aimed at technical/product founders plus their sales, marketing & product executives who are responsible for the go-to-market strategy for their company.

Zero to 50m

David Skok

Go Viral on the Social Web: The Definitive How-To guide!

XPLAIN

Fully Automated Link Building - Brighton SEO.pdf

Sam Oh

Carlos Meza BrightonSEO 2023 Is content king in modern SEO.pptx

Carlos Meza

How to develop a product line presentation from the Digital Retailers Association Product Line Building Workshop. These materials cover interactive exercises to develop product lines using Lateral Thinking, Product Category Differentiation, Value Curve for building products, the Product Line Grid, The New Product Canvas, Product Funnel Grid, Product Innovation Mapping, Leverage in Product Development, Trend-Driven Product Development, Kaizen and Product Pivots,

Product Development How To Develop A Product Line

Roland Frasier

What's hot (20)

Get inside your Buyers Head - Improve Funnel Conversion Rates

Why Startups Suck at Marketing

Content Marketing Lessons From 10 Years and 41,525,103 Website Visits

Building a sales & marketing machine

eCommerce Internal Linking - Into the Spider-Verse (BrightonSEO edition)

Slash | How to build a B2B sales machine

128 High Converting Growth Hacks - the most epic growth hacking list

A Playbook for Achieving Product-Market Fit

Mobile Growth: Best Strategies, Tools and Tactics

Content writers: will AI take your job?

The Art & Science of Growth Hacking

Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...

JTBD for Marketering - Nopadon.pdf

From Zero to a Million Users - Dropbox and Xobni lessons learned

Content Marketing in 15 Minutes a Day for Restaurants

Zero to 50m

Go Viral on the Social Web: The Definitive How-To guide!

Fully Automated Link Building - Brighton SEO.pdf

Carlos Meza BrightonSEO 2023 Is content king in modern SEO.pptx

Product Development How To Develop A Product Line

Similar to High Tempo Testing

The goal of any company with a valuable product is to scale adoption by qualified customers. This generally requires establishing a "north star metric" and managing growth against that metric. These slides show how you can build a core growth team that coordinates the efforts of the full company to drive sustainable growth. It is important that every idea be treated as a test. The more tests you run the more learning you gain for growing the company. But running a lot of tests requires a process that is explained in the slides.

Building the Ultimate Full Company Growth Team

Sean Ellis

"Building a Company Wide Growth Culture" at SaaStr Annual 2016

saastr

3 Essential Coaching Skills

sparkagility

3 Essential Coaching Skills for Project Leaders

Salah Elleithy

Presented at Flowcon SF on Nov 1, 2013 Nothing interrupts the continuous flow of value like bad surprises that require immediate attention: major defects; service outages; support escalations; or even scrapping just-completed capabilities that don't actually meet business needs. You already know that the sooner you can discover a problem, the sooner and more smoothly you can remedy it. Agile practices involve testing early and often. However feedback comes in many forms, only some of which are traditionally considered testing. Continuous integration, acceptance testing with users, even cohort analysis to validate business hypotheses are all examples of feedback cycles. This talk examines the many forms of feedback, the questions each can answer, and the risks each can mitigate. We'll take a fresh look at the churn and disruption created by having high feedback latency, when the time between taking an action and discovering its effect is too long. We'll also consider how addressing "bugs" that may not be detracting from the actual business value can distract us from addressing real risks. Along the way we'll consider fundamental principles that you can apply immediately to keep your feedback cycles healthy and happy.

On the Care and Feeding of Feedback Cycles

Elisabeth Hendrickson

Awl ass intro mtg v2

shaunallison

Recorded webinar: http://slidesha.re/1nOR5i5 Subscribe: http://ksmartin.com/subscribe Purchase the book: http://bit.ly/TOObk This webinar features content from Karen's workshop and talk at the Lean Enterprise Institute and Lean Frontiers Coaching Summit, held on July 29 & 30, 2014 in Long Beach, California. Both the workshop and talk focus on learning how to break the "telling" habit as a leader or improvement coach, and how to use the right questions at the right time to develop people more effectively and get better work results.

Coaching: Asking the Right Questions

TKMG, Inc.

Interview workshop slides final

Wendy Preisman Turell, DrPH

Few organizations feel they are reaching their true potential. Technical debt, antiquated processes, outdated measurements, and a never-ending influx of new requirements leave most feeling incapable of doing much more than keeping the lights on. Overcoming these challenges and achieving meaningful change doesn't happen overnight, but is a necessity in order to create the next generation of IT operations and create the foundation for DevOps success. This session will explore techniques that modern IT organizations are adopting to become more agile, adopt new technologies like containers, new processes like DevOps, and build a culture of continual improvement. We will touch on how to: • Improve responsiveness, collaboration, and learning across teams • Adopting and supporting new technologies like containers, microservices, and automation • Gain more visibility into your environments and processes to identify constraints • Grow beyond firefighting and just getting the next feature out

Adopting a Continual Improvement Mindset for IT

Josh Atwell

Growth Hacking Paris with Sean Ellis from GrowthHackers.com!

TheFamily

Systematic reviews and social media

Anne Littlewood

ACI design and creativity slides 2019 day4

R. Sosa

Making your research and teaching more efficient, transparent and impactful

Jay Van Bavel

How to ace the viva dcu

DoctoralNet Limited

UX Café October 2013: Eewei Chen '101 design ingredients to solve big tech pr...

UX Café

DATA COLLECTION.pptx

Suzie De Leon

Sue Sheerin: Why self-assessment exciting?

eaquals

Creativity for Learning 5: Linking theory and practice through collaborative ...

Chrissi Nerantzi

Questioning Techniques

Paul Redford

Action research

Isobel Dunbar

Similar to High Tempo Testing (20)

Building the Ultimate Full Company Growth Team

"Building a Company Wide Growth Culture" at SaaStr Annual 2016

3 Essential Coaching Skills

3 Essential Coaching Skills for Project Leaders

On the Care and Feeding of Feedback Cycles

Awl ass intro mtg v2

Coaching: Asking the Right Questions

Interview workshop slides final

Adopting a Continual Improvement Mindset for IT

Growth Hacking Paris with Sean Ellis from GrowthHackers.com!

Systematic reviews and social media

ACI design and creativity slides 2019 day4

Making your research and teaching more efficient, transparent and impactful

How to ace the viva dcu

UX Café October 2013: Eewei Chen '101 design ingredients to solve big tech pr...

DATA COLLECTION.pptx

Sue Sheerin: Why self-assessment exciting?

Creativity for Learning 5: Linking theory and practice through collaborative ...

Questioning Techniques

Action research

More from Sean Ellis

Focusing on the right aggressive growth goals can dramatically increase your company's overall growth rate, moving your team from random ideation to creative problem solving. In these slides, Sean explains how to set and achieve high impact growth goals. You’ll learn how to determine the ideal target and time frame for each growth goal and how to rally your team around a proven growth hacking process for achieving the goals.

Setting and Achieving Growth Goals

Sean Ellis

Product Driven Growth from Lean Product Meetup

Sean Ellis

SaaSFest 2015: Accelerating Organic Growth Through High Tempo Testing

Sean Ellis

Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...

Sean Ellis

Product Meets Growth - How Product Teams Are Key To Driving Sustainable Growth

Sean Ellis

Annual marketing plans can't keep up with the rapidly changing landscape of digital marketing acquisition channels and tactics. To remain relevant, CMOs and marketing teams need to completely rethink their marketing/growth approach. This presentation highlights the agile process that today's fastest growing companies are adopting - from building the right team to executing an agile marketing process.

Agile Marketing Meetup: Moving Beyond the Marketing Plan So You Remain Relevant

Sean Ellis

Busting the Myth of Growth Hacking Magic

Sean Ellis

Growth Hacking with Data: How to Find Big Growth with Deep Data Dives

Sean Ellis

Growth Hacking: The Human Operating System for Martech (Marketing Technology)

Sean Ellis

What Made WhatsApp Worth $19 Billion?

Sean Ellis

Unlocking Growth: Building a sustainable growth engine with the new rules of...

Sean Ellis

From social media to website analytics, there are literally hundreds of things you could track, measure and try to improve. But what is the one metric that if improved would mean a big win for your business? This deck will help you hone in on the metrics that really matter to your business. You’ll get their insights on the tools and strategies you need to find, prioritize and grow the numbers that will result in big wins for your business.

Finding the one growth metric that matters

Sean Ellis

Grow with CRO - 7 Ways CRO is a Growth Driver for Your Business

Sean Ellis

Six Growth Hacks to Get More Conversions from Your Content Marketing

Sean Ellis

Startups require a very different marketing approach than established companies. This slide deck from the first marketer at Dropbox, LogMeIn, Lookout and others explains how you prioritize your marketing efforts in a startup. It starts by give you a way to determine startup’s growth potential or if it's even growable at all. It then explains why stacking the odds is critical for reaching your startup’s growth potential and tells you how to stack the odds for growth. Finally it shares a case study explaining how a startup overcame growth frustrations to become worth more than one billion dollars today.

Startup Marketing Conference - Stacking the Odds for Authentic, Sustainable G...

Sean Ellis

The Biggest Growth Opportunity is Right Under Your Nose

Sean Ellis

Pioneers festival preso

Sean Ellis

Stacking the Odds for Authentic Growth

Sean Ellis

Startup Marketing Slides from Lean Startup Circle LA

Sean Ellis

Berkeley-Columbia Executive MBA Program (Steve Blank)

Sean Ellis

More from Sean Ellis (20)

Setting and Achieving Growth Goals

Product Driven Growth from Lean Product Meetup

SaaSFest 2015: Accelerating Organic Growth Through High Tempo Testing

Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...

Product Meets Growth - How Product Teams Are Key To Driving Sustainable Growth

Agile Marketing Meetup: Moving Beyond the Marketing Plan So You Remain Relevant

Busting the Myth of Growth Hacking Magic

Growth Hacking with Data: How to Find Big Growth with Deep Data Dives

Growth Hacking: The Human Operating System for Martech (Marketing Technology)

What Made WhatsApp Worth $19 Billion?

Unlocking Growth: Building a sustainable growth engine with the new rules of...

Finding the one growth metric that matters

Grow with CRO - 7 Ways CRO is a Growth Driver for Your Business

Six Growth Hacks to Get More Conversions from Your Content Marketing

Startup Marketing Conference - Stacking the Odds for Authentic, Sustainable G...

The Biggest Growth Opportunity is Right Under Your Nose

Pioneers festival preso

Stacking the Odds for Authentic Growth

Startup Marketing Slides from Lean Startup Circle LA

Berkeley-Columbia Executive MBA Program (Steve Blank)

High Tempo Testing

1. Sean Ellis @seanellis High Tempo Testing Moving Beyond Hustle to Building a Predictable Growth Machine

2. Sean Ellis @seanellis 3,2,1... Hustle!

3. Sean Ellis @seanellis Early Growth: Twitter Focused Feedback: “I find growth articles on Twitter” Followers: Easy to find people that tweet growth articles Content: Long tail of content ideal for Twitter

4. Sean Ellis @seanellis Goal: Keep Conversation Going

5. Sean Ellis @seanellis Some Vacations...

6. Sean Ellis @seanellis Viability Achieved by Month 3

7. Sean Ellis @seanellis Goals Not Met After First Year 3 Flat Months!

8. Sean Ellis @seanellis “Inputs are what you actually control.” Andrew Chen

9. Sean Ellis @seanellis Problem: Limited Testing Had specific growth goals, but just said “test more”. Team was too busy for much testing... Building product Executing Twitter Playbook Engaging with community

10. Sean Ellis @seanellis Solution: High Tempo Testing Launch 3 Tests Per Week (Clear “input” goal)

11. Sean Ellis @seanellis Result... Started high tempo testing

12. Sean Ellis @seanellis Hustle › Growth Process

13. Sean Ellis @seanellis High Tempo Testing Process Build Knowledge Base Unbridled Ideation Prioritize Backlog High Tempo Testing

14. Sean Ellis @seanellis Challenges of High Tempo Testing 1. Enough ideas: Ideas are fuel for tempo 2. Prioritizing ideas: Finding best ideas to test 3. Implementing tests: Getting the resources 4. Accurate testing: Instrumenting & analyzing 5. Avoiding repeats: Not running same tests

15. Sean Ellis @seanellis Challenge 1: Enough Ideas Unbridled ideation from full team Ideas across all vectors Individual & Team Accountability Acqusition Activation Retention Revenue Referral

16. Sean Ellis @seanellis Challenge 2: Prioritizing Ideas Formalize ideas in “Experiment Doc” Include research, hypothesis, target lever... Score on impact, confidence & ease

17. Sean Ellis @seanellis Challenge 3: Implementing Ideas Weekly growth meeting to plan/assign current week’s tests. Accountability for previous week. Recommend Autonomous growth team with dedicated development, design, analysis... Empower less technical team members w tools

18. Sean Ellis @seanellis Challenge 4: Accurate Testing Read Peep Laja (Next presenter)

19. Sean Ellis @seanellis Challenge 5: Avoiding Repeats Capture all tests in Knowledge Base Ensure knowledge base shared, clear & searchable Worked Failed Inconclusive

20. Sean Ellis @seanellis High Tempo Test Requirements Process People System Build Knowledge Base Unbridled Ideation Prioritize Backlog High Tempo Testing

21. Sean Ellis @seanellis High Tempo Testing Process Build Knowledge Base Unbridled Ideation Prioritize Backlog High Tempo Testing

22. Sean Ellis @seanellis Winning Test Example via Process Moved email collector from bottom to top. Result: 700% increase in collected emails Impact prediction was 4/10

23. Sean Ellis @seanellis High Tempo Testing Advantages 1. Predictable Growth 2. Scaleable Growth 3. Easier Hiring 4. Address Bottlenecks

24. Sean Ellis @seanellis Thank you! @seanellis

High Tempo Testing

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to High Tempo Testing

Similar to High Tempo Testing (20)

More from Sean Ellis

More from Sean Ellis (20)

High Tempo Testing