An Open Standard for Learnable APIs

CTO, New Iron
Janelle Klein
Learnable APIs
An Open Standard
for
Easy to Learn and Use Hard to Learn and Use
Learnability Index
©2015 New Iron Group
Saturday, August 8, 15

RESET
Why should I care?

RESET
...
The Modern API Decision-Making Process
“Hmm... I need a Contact Management API.”

RESET
Contact Management API
...

RESET
...
Our API directory now includes 63 CRM APIs. The newest is thePingGiant API. The most
popular, in terms of mashups, is theSalesforce API. We list 44 Salesforce mashups. Below
you'll ﬁnd some more stats from the directory, including the entire list of CRM APIs.
source: http://programmableweb.com

RESET
...
“Ugh...this is taking too long.”

RESET
...
“Hey George, what API do you use for Contact Management?”

RESET
...
“We made our own Salesforce mashup.
I’ll send you a link.”

RESET
...
“We made our own Salesforce mashup.
I’ll send you a link.”
“Thanks, George. I’ll use that one I guess.”

RESET
Okay... but this is just a minor annoyance.
source: Domics - “What’s a Shrug?”
The real pain happens later...

RESET
https://www.youtube.com/watch?v=0mUN3RppEHE
Fear and Inspiration
Josh Corman, DevOpsDays Austin 2015

Infecting
We’re putting software and connectivity in everything...

The security implications are frightening...
100 million lines of code running a car!

RESET
90% of our software is assembled from existing parts...
The Complexity and Risk is in the Supply Chain

The Talent Shortage
Higher Demand for Software
Software Development knowledge is mostly tacit knowledge
that’s learned through mentorship (not school)
Broken Education System+
More Incompetent Devs building software

Bad Software is like Environment Pollution
Breaking
Point
Cost
&
Risk
Complexity
Cumulative Complexity in the SW Supply Chain
(i.e. shared dumping ground of crappy software)
The risk to public safety is on all of us.

RESET
What Can We Possibly Do?
source: Domics - “What’s a Shrug?”

RESET
The Difference? Modern Building Codes
Haiti Quake: 7.0 with 230,000 deaths
Chile Quake: 8.8 with 279 deaths

RESET
We’ve copied everything from Deming’s manufacturing ideas
except Quality and Risk Management Practices.

RESET
To conquer the challenges of quality and risk
We have to manage quality in the Supply Chain

The @iamthecavalry community rally call:
It’s not our software, but it is our problem.

Let’s take ownership of the supply chain problems
and design the future we want.

RESET
Deliberate Design of Human Systems
There is always cost. There are always consequences.
Open != Good
Private Competition
Survival of the Fittest
Self-Optimizing
Creativity
Innovation
Open Standards
Define the Competition
Optimize the Whole
Public Debate
Empathy
Interactions
cause
system effects

Let’s Take Ownership of Evolution
Pain Pushes
(Threat of Survival)
Opportunity Pulls
(Path of Least Resistance)
“The System”

Learnable APIs
Open Standard
for
What if we had...

Mistakes are primarily
caused by a lack of understanding.
Learnability Index
Learnability measures the cost of understanding.
Learnability measures the risk of mistakes.

RESET
...
Let’s Try this Again...

RESET
...
Quality decisions create selection pressures for quality in the market

RESET
...
Quality decisions create selection pressures for quality in the market
Cost of Ownership Analysis

What information do we need
to make quality supply chain decisions?
What’s the strategy to get there?

RESET
...
I’ve been working on a related challenge
for the last ~8 years.
Software Rewrite Cycle
How to Break
the
Start%
Over%
Unmaintainable%
So0ware%

RESET
My software project...
We tried to do all the “right” things
...and brought down production 3x in a row.

Technical Debt Risk of Mistakes
I thought the problem was
Technical Debt

Risk of Mistakes
?
Most of our mistakes were in the
most well-written parts of the code.

We made significantly more mistakes
in code that we didn’t write ourselves.
Lower
Familiarity
More
Mistakes=
There had to be more to the story...

Complex(
So*ware(
...pain...
This is what I knew...
What made development feel painful?

Unexpected
Behavior
Problem
Resolved
Tracking Painful Interaction with the Code (Friction)
Troubleshooting
Progress
5 hours and 18 minutes of troubleshooting...
PAINFUL

The Amount of PAIN was Driven By...
Likeliness(of((
Unexpected(
Behavior(
Cost(to(Troubleshoot(and(Repair(
High(Frequency(
Low(Impact(
Low(Frequency(
Low(Impact(
Low(Frequency(
High(Impact(
PAIN(

What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
What causes PAIN?
Bad Input Assumptions
Stale Memory Mistakes
Semantic Mistakes
Similarity Mistakes
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Generation
Using Debugger
Most of the pain was caused by human factors.

What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
What causes PAIN?
Bad Input Assumptions
Stale Memory Mistakes
Semantic Mistakes
Similarity Mistakes
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Most of the pain was caused by human factors.
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Generation
Using Debugger

“Idea Flow” is a metaphor for the
human interaction in software development
Idea Flow Map

Conﬁrmed
Expectations
Conﬁrmation:
Everything feels as it should be...

Violated
Expectations
Conﬂict:
Something is out of place...

The Idea Flow Process looks like this:

“Friction” occurs in the context of a Process.
Pain is caused by high-friction movement.
friction

Friction occurs in Three Places:
ConflictConfirm
Rework'
Learn'
Validate(
Modify'
Conﬁrming Loop! Conﬂict Loop!
Troubleshoot'

Troubleshooting Pain
Likelihood)of))
Unexpected)
Behavior)
Cost)to)Troubleshoot)and)Repair)
High)Frequency)
Low)Impact)
Low)Frequency)
Low)Impact)
Low)Frequency)
High)Impact)
PAIN)

Learning Pain
Likelihood)of))
working)with)
Unfamiliar)
Code)
Cost)to)Learn)
High)Frequency)
Easy)to)Learn)
Low)Frequency)
Easy)to)Learn)
Low)Frequency)
Hard)to)Learn)
PAIN)

Rework Pain
Likelihood)of))
making)a))
Bad)Assump4on)
Cost)to)Correct)Decisions)
High)Uncertainty)
Low)Delay)
Low)Uncertainty)
Low)Delay)
Low)Uncertainty)
High)Delay)
PAIN)

How do we measure
Idea Flow in Developer Experience?

Idea Flow Mapping Tools
(Open Source)

Troubleshooting
Progress
Learning
Rework
(Open Source)

Troubleshooting
Progress
Learning
Rework
(Open Source)
Experiment via
Unit Testing
Experiment via
Running App

Troubleshooting
Progress
Learning
Rework
(Open Source)
Tags: #problemtag

Input:
Task + Constraints
Target: Optimal Idea Flow
Output: Actual Friction
1.
Visibility
2.
Clarity
3.
Awareness
Idea Flow Learning Framework
1. Visibility - Make the Pain Visible
2. Clarity - Understand the Causes of Pain
3. Awareness - Improve Decision Habits
(data-driven learning)

We can learn to read the
Visual Indicators in Idea Flow Maps
Le#$Atrium$
Le#$Ventricle$
Right$Ventricle$
Right$Atrium$
What’s$causing$this$pa7ern?$
Similar to how an EKG helps doctors diagnose heart problems...

...Idea Flow Maps help developers diagnose software problems.
Problem-Solving
Machine

= Solution Strategy
The “Heart” of Software Development
(the problem-solving machine)

= Solution Strategy
Evaluate alternatives and
decide on a general strategy

= Solution Strategy
Scan through the code and
build a conceptual model of
how it works

= Solution Strategy
Feedback loop phase involves writing a
little code, validating that it works, then
reﬁning the details of the strategy.

= Solution Strategy
If we discover our strategy isn’t going to
work, we have to go back to the
drawing board to find an alternative.

Depending on where the disruptions are in the process,
we see a different effect in Idea Flow.

"How did you evaluate the possible options and choose a strategy?"
"What was wrong with the different strategies you tried?"
"What was the discovery that made you choose a different direction?"
Problems with Evaluating Alternatives

"Were you working with something that you were unfamiliar with?"
"Did you run into code that was noisy, ambiguous, or misleading?"
"What do you think made it difficult to learn?"
Problems with Modeling

"Did your task involve changes to complex code or business rules?"
"Were there a lot of details that you had to keep in your head?"
"What was causing the complexity in the validation cycles?"
Problems with Reﬁning

"What experiments did you run to troubleshoot the problem?"
"How many times did you run the experiment?"
"How long did it take to get through each experiment cycle?"
Problems with the Validation Cycle

"Was there something in the code that made these changes especially mistake-prone?"
"How familiar were you with the language and tools you were working with?"
"Were you tired or distracted when you did the work?"
Problems with Execution

The Ten Pains of Software Development

Add up Friction by Pain Type
100 hours
50 hours
Troubleshooting
Learning
Rework
Focused Improvements Visible Feedback

What’s Causing the
Software Rewrite Cycle?
Start%
Over%
Unmaintainable%
So0ware%

Cycle of Escalating Risk
Constant'
Urgency'
Time'
Pressure'
Ignore'
Risk'
Increase'
Likeliness'
&'Magnitude'
of'Pain'
More'Frequent'
Bugs'and'
Higher'
Task'Eﬀort'

Case Study 1:
Healthy project about 10 months old
Troubleshooting
Progress
Learning
Rework10-20% friction
Effects of Escalating Risk

Case Study 2:
Thrashing project about 18 months old
Troubleshooting
Progress
Learning
0:00 28:15
12:230:00

Case Study 3:
Post-meltdown project about 12 years old
Troubleshooting
Progress
Learning
7:070:00
0:00 19:52

Case Study 1
Case Study 2
Case Study 3
1 day
2 days
1 day
3 days
1 day
3 days
We can’t see these effects by measuring velocity or task lead-time.

RESET
Remember This?
We’ve copied everything from Deming’s manufacturing ideas
except Quality and Risk Management Practices.

Idea Flow Learning Framework
is a software control system!
Input:
Task + Constraints
Target: Optimal Idea Flow
Output: Actual Friction
1.
Visibility
2.
Clarity
3.
Awareness
My background...
Statistical Process Control (SPC) and Supply Chain in Manufacturing

How do we measure
Learnability of an API?

“Learnability” is the average friction across
a specific set of Developer Use Case Experiments
Learnability Index
Open Standard for All APIs and Libraries

RESET
...
“How do we get here?”
The Conversation Starts with a Vision
Cost of Ownership Analysis
(Even if it’s a half-baked vision)

Proposal for a Road Map
Open Mastery (Launching Fall 2015)
Developer peer learning network focused on using
data-driven learning to codify open decision models
and standards for industry-wide problems.
Community Backlog
1. Personal Mastery (Codify Idea Flow-based Software Control decision-making)
2. Open Software Mastery Certification
3. Team Mastery (Software Mentorship, Knowledge Integration, Improvement Strategy)
4. Open Software Mentorship Certification
5. Supply Chain Mastery (Cost of Ownership Analysis, Component Decision Hub)
6. Open API DX Certification

Think about it,
Blog about it,
Tweet about it!
@janellekz
#ideaﬂow!
Free e-book if you sign up
by Aug 30th!
@janellekz
janelle@newiron.com
Twitter:
Email:

An Open Standard for Learnable APIs

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

Similar to An Open Standard for Learnable APIs

Similar to An Open Standard for Learnable APIs (20)

More from Arty Starr

More from Arty Starr (14)

Recently uploaded

Recently uploaded (20)

An Open Standard for Learnable APIs