No (Lab) Jacket Required [Agile Midwest Conference]
•
0 likes•396 views
Slides as presented at the 2019 Agile Midwest Conference. https://agilemidwest.org Abstract: Hypothesis-Driven Development is thinking about the development of new ideas, products and services – even organizational change – as a series of experiments to determine whether an expected outcome will be achieved, so we need to know how to design and run experiments properly. This session helps participants understand the importance of using experiments to help teams and organizations learn and improve, while giving hands-on practice in designing experiments to yield measurable evidence for that learning.
Recommended
Recommended
More Related Content
More from Matthew Philip
More from Matthew Philip (20)
Recently uploaded
Recently uploaded (20)
No (Lab) Jacket Required [Agile Midwest Conference]
- 1. Matt Philip NO (LAB) JACKET REQUIRED: DESIGNING EXPERIMENTS FOR LEARNING @mattphilip #AgileMW2019
- 4. In complex environments, you can’t follow recipes or conduct detailed analysis to understand the situation. Rather, you must experiment (probe). -- Cynefin Improve collaboratively, evolve experimentally. -- Kanban Method Pivoting …. a structured course correction designed to test … hypothesis about the product, strategy, and engine of growth. -- Lean Startup Each feature includes a benefit hypothesis. -- SAFe WHYEXPERIMENTS? @mattphilip #AgileMW2019
- 5. ELEUSISEXPEDITIOUS A GAME ABOUT HYPOTHESES @mattphilip #AgileMW2019
- 6. IDEA ◆ This is a variation on Eleusis Express; a card game of inductive reasoning which itself is a modification of Robert Abbott’s game Eleusis. ◆ One player has a secret rule for which cards can be played. ◆ For example: Each card has to be a different color from the card before it. Other players lay down cards to discover the pattern. If a player lays down a card correctly, he or she can try to guess the rule. @mattphilip #AgileMW2019
- 7. OBJECT ◆ A game consists of one or more rounds/hands. ◆ A different player is chosen as dealer of each new round, and it is the dealer who knows the secret rule. The dealer does not play a hand that round. ◆ All plays are made to a central layout that grows as the round progresses. A layout consists of a horizontal mainline of correct cards that follow the secret rule. Below the mainline are vertical sidelines of mistake cards that did not follow the rule. These sideline cards can be overlapped to save space. @mattphilip #AgileMW2019
- 8. DEALER ◆ The dealer is given a secret rule. ◆ He or she places the pile of cards face down, turns over the top card, and puts it on the table. This will be the start of the mainline. ◆ Before play starts, the dealer may give a hint about the rule. ◆ The player to the left of the dealer goes first, then the play continues around to the left. @mattphilip #AgileMW2019
- 9. PLAYERS ◆ A player turns the top card from the deck and lays it on the table. He or she asserts whether it is correct or not, and the dealer confirms or denies. ◆ If correct, the card goes to the right of the last card on the mainline. ◆ If incorrect, the card goes below the last card (it either starts a sideline or it adds to a sideline). @mattphilip #AgileMW2019
- 10. PLAYERS (NO PLAY) ◆ A player has the option of declaring that they have no correct or incorrect cards to play. They shows their hand (to everyone) and the dealer says whether the player is right or not. ◆ If the player is wrong, the dealer chooses one correct or incorrect card from their hand which could have been played, and puts it on the layout. For example if the player declares they have no correct cards, a correct card would be chosen. The player keeps their hand and they must draw one card from the stock. ◆ If the player is right, and their hand is down to one card, that card is put in the stock and the round is over. If they have more than one card, the dealer counts their cards and puts them on the bottom of the stock. They then deal the player a new hand, but with one less card. @mattphilip #AgileMW2019
- 11. GUESSING ◆ Whenever a player makes a correct assertion, he or she can guess the rule. Everyone must hear the guess. ◆ The dealer then says whether the player is right or wrong. If wrong, the game continues. If right, the round ends. @mattphilip #AgileMW2019
- 14. 1. Choose who is who. One player becomes the codemaker, the others are the codebreakers 2. Choose 4 digits. The codemaker chooses a pattern of four digits; for example 1 2 3 4. 3. Try guessing the pattern. The codebreakers try to guess the pattern, in both order and number, within twelve turns. Each guess is made by writing a row of four . 4. Provide feedback. After each guess the codemaker provides feedback by placing from zero to four key dots next to the guess. 1. A full key dot [●] indicates the existence of a code number which is correct in both number and position. 2. An empty dot [○] indicates the existence of a correct number placed in the wrong position. 5. Keep making guesses. Once feedback is provided, another guess is made. Guesses and feedback continue until either the codebreakers guess correctly, or twelve incorrect guesses are made. INSTRUCTIONS @mattphilip #AgileMW2019
- 15. As before with the following additions: • Before a guess. Write down the intent of the guess – what it is you want to learn. • After feedback. Write down the result and what you actually learned. WORKSHEET @mattphilip #AgileMW2019
- 17. • Define a question • Gather information and resources (observe) • Form an explanatory hypothesis • Test the hypothesis by performing and experiment and collecting data in a replicable manner • Analyze the data • Interpret the data and draw conclusions that serve as a starting point for a new hypothesis • Publish results SCIENTIFICMETHOD @mattphilip #AgileMW2019
- 20. HOW MUCH LEARNING? @mattphilip #AgileMW2019
- 23. WHAT IS “DONE” FOR YOU? -- John Cutler @mattphilip #AgileMW2019
- 26. • Company policy to give a bonus to employees who refer new hires • Implementing SAFe • Rolling out a new team-leadership model • Work-from home policy • [Your own example] HOWWOULDYOUTESTTHESE? @mattphilip #AgileMW2019
- 27. • Psychological safety to fail • Learning is outcome • Time-based • Measurable • Single variable • Provable (and disprovable) hypothesis ELEMENTSOFGOODEXPERIMENTS @mattphilip #AgileMW2019
- 29. • Language: “Fail Fast” à “Prove/disprove hypothesis” • Psychological (and other) Safety • Leaders: Establish the boundaries (SETT) • Organizational experiments board • “Goldratt User Stories” • Finding your optimal WIP limit • A/B tests • Retrospective actions OTHERCONSIDERATIONS @mattphilip #AgileMW2019
- 32. SOURCES AND RESOURCES ◆ Karl Scotland (https://availagility.co.uk/) ◆ https://yorkesoftware.com/2018/03/09/goldratt-user-stories/ ◆ http://cognitive-edge.com/methods/safe-to-fail-probes/ ◆ Eleusis Express http://www.logicmazes.com/games/eleusis/express.html ◆ Freakonomics Radio: 370. How to Fail like a Pro ◆ Unlearn by Barry O’Reilly ◆ Agile BOSSA nova book ◆ How to Measure Anything ◆ John Cutler ◆ The Surprising Power of Online Experiments (https://hbr.org/2017/09/the-surprising-power- of-online-experiments) ◆ https://exp-platform.com/Documents/2015- 08OnlineControlledExperimentsKDDKeynoteNR.pdf ◆ http://www.theagilefactor.com/ ◆ https://barryoreilly.com/how-to-implement-hypothesis-driven-development/ ◆ http://mattphilip.wordpress.com @mattphilip #AgileMW2019