Sean Ellis, Founder & CEO, Qualaroo

1. Sean Ellis @seanellis
High Tempo Testing
Moving Beyond Hustle to Building
a Predictable Growth Machine

2. Sean Ellis @seanellis
3,2,1... Hustle!

3. Sean Ellis @seanellis
Early Growth: Twitter Focused
Feedback: “I find growth articles
on Twitter”
Followers: Easy to find people that
tweet growth articles
Content: Long tail of content ideal
for Twitter

4. Sean Ellis @seanellis
Goal: Keep Conversation Going

5. Sean Ellis @seanellis
Some Vacations...

6. Sean Ellis @seanellis
Viability Achieved by Month 3

7. Sean Ellis @seanellis
Goals Not Met After First Year
3 Flat Months!

8. Sean Ellis @seanellis
“Inputs are what you
actually control.”
Andrew Chen

9. Sean Ellis @seanellis
Problem: Limited Testing
Had specific growth goals, but just said
“test more”.
Team was too busy for much testing...
Building product
Executing Twitter Playbook
Engaging with community

10. Sean Ellis @seanellis
Solution: High Tempo Testing
Launch 3 Tests Per Week
(Clear “input” goal)

11. Sean Ellis @seanellis
Result...
Started high
tempo testing

12. Sean Ellis @seanellis
Hustle › Growth Process

13. Sean Ellis @seanellis
High Tempo Testing Process
Build
Knowledge
Base
Unbridled
Ideation
Prioritize
Backlog
High
Tempo
Testing

14. Sean Ellis @seanellis
Challenges of High Tempo Testing
1. Enough ideas: Ideas are fuel for tempo
2. Prioritizing ideas: Finding best ideas to test
3. Implementing tests: Getting the resources
4. Accurate testing: Instrumenting & analyzing
5. Avoiding repeats: Not running same tests

15. Sean Ellis @seanellis
Challenge 1: Enough Ideas
Unbridled ideation from full team
Ideas across all vectors
Individual & Team Accountability
Acqusition
Activation
Retention
Revenue
Referral

16. Sean Ellis @seanellis
Challenge 2: Prioritizing Ideas
Formalize ideas in “Experiment Doc”
Include research, hypothesis, target lever...
Score on impact, confidence & ease

17. Sean Ellis @seanellis
Challenge 3: Implementing Ideas
Weekly growth meeting to plan/assign current
week’s tests. Accountability for previous week.
Recommend Autonomous growth team with
dedicated development, design, analysis...
Empower less technical team members w tools

18. Sean Ellis @seanellis
Challenge 4: Accurate Testing
Read
Peep Laja
(Next presenter)

19. Sean Ellis @seanellis
Challenge 5: Avoiding Repeats
Capture all tests in Knowledge Base
Ensure knowledge base shared, clear & searchable
Worked Failed Inconclusive

20. Sean Ellis @seanellis
High Tempo Test Requirements
Process People System
Build
Knowledge
Base
Unbridled
Ideation
Prioritize
Backlog
High
Tempo
Testing

21. Sean Ellis @seanellis
High Tempo Testing Process
Build
Knowledge
Base
Unbridled
Ideation
Prioritize
Backlog
High
Tempo
Testing

22. Sean Ellis @seanellis
Winning Test Example via Process
Moved email collector
from bottom to top.
Result: 700% increase
in collected emails
Impact prediction was
4/10

23. Sean Ellis @seanellis
High Tempo Testing Advantages
1. Predictable Growth
2. Scaleable Growth
3. Easier Hiring
4. Address Bottlenecks

24. Sean Ellis @seanellis
Thank you!
@seanellis