Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Honest Experimentation by Jonathan Bertfield

Honest Experimentation by Jonathan Bertfield @Agile Israel 2019

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Honest Experimentation by Jonathan Bertfield

  1. 1. HONEST EXPERIME NTATION JONATHANBERTFIELD @BERTERS @MYSPINNAKERCO
  2. 2. @BERTERS @MYSPINNAKERCO JONATHAN@MYSPINNAKER.CO
  3. 3. why we experiment
  4. 4. courage to learn
  5. 5. courage to be transparent
  6. 6. traditional execution reality
  7. 7. modern execution reality
  8. 8. lean startup execution reality
  9. 9. experiment principles SMALL BATCHES MOVING FROM DOUBT TOWARDS CERTAINTY
  10. 10. continuous experimentation
  11. 11. experiment calculus LIKELIH OOD OF SUCCE SS QUANTITY OF LEARNING QUALITY OF LEARNING X=
  12. 12. what we get from experiments
  13. 13. from hypothesis to experiment MEAS URE BUIL D LEAR N What do we need to build to generate the data we can measure What data will allow us to measure this learning What do we need to learn to validate or invalidate our riskiest assumption
  14. 14. what is a hypothesis? “If then” statement that helps design tests for an assumption Clarifies your current understanding of what uncertainty you seek to resolve Is specific in the action, timing and value / amount of impact
  15. 15. Assumption vs. Hypothesis “I believe that better photos of the listings will improve the visibility and desirability of those listings when compared to those with owner-generated photos” ——————————————————— "If we take professional photos for listings, those listings will get 2-3 times more business than others in the first 60 days of the experiment.” Assumption Hypothesis
  16. 16. experiment structure ▸The hypothesis we believe to be true ▸What we will do to test the hypothesis ▸What will we measure to validate or invalidate the hypothesis ▸What threshold for success will we look to
  17. 17. experiment structure ▸We believe that urban vehicle owners in India place a high value on saving time when refueling and will sign up for auto payment so they can refuel quicker. ▸We will set up an express lane for scooters / motorbikes at 1 filling station for 1 day to simulate an auto payment sign up and payment process ▸We will measure how many riders participate in the sign up process and how long it takes to convert each rider ▸We will be successful if we can convince 25% of riders to participate and take an average of less than 3 minutes to convert riders
  18. 18. concierge mvp
  19. 19. concierge mvp
  20. 20. mvp case study CONDUIT CASE STUDY US DEPT OF DEFENSE
  21. 21. Problem interview - Card Sorting The team sends more relevant products. The team finds products from fewer sites. The team spends less time reformatting. The team sends more timely products. The team spends more time interacting with briefers. The team spends less time on daily product dissemination, more time on liaison work The team is more involved in daily product dissemination. The team targets product dissemination by topic area. The team misses fewer products in "the funnel." The team sends more relevant products. The team finds products from fewer sites. The team spends less time reformatting. The team sends more timely products. The team spends more time interacting with briefers. The team spends less time on daily product dissemination, more time on liaison work The team is more involved in daily product dissemination. The team targets product dissemination by topic area. The team misses fewer products in "the funnel." our hypotheses… the users… What do our users value most?
  22. 22. Paper sketch “We love the feeds!!!”
  23. 23. Non clickable prototype
  24. 24. Clickable prototype
  25. 25. experiment structure ▸We believe that a wide community of DoD analysts will value the ability to quickly synthesize content into briefing ready reports ▸We will run an publicity campaign on internal DoD web channels to drive traffic to a landing page where we will capture interest ▸We will measure views and signups for pilot ▸Success will be 2000 views and 20 sign ups within 1 week
  26. 26. Landing page + explainer video
  27. 27. experiment artifact
  28. 28. 3D modelling
  29. 29. value pitch
  30. 30. Thank you JONATHANBERTFIELD JONATHAN@MYSPINNAKER.CO WWW.MYSPINNAKER.CO bit.ly/leadinginnovationIs Join us tomorrow in Herzilya for a full day workshop on leading innovation

×