This document discusses machine learning product development for industrial IoT applications. It presents a spiral model for an ML product development cycle consisting of plan, build, test, learn, and release phases. It then discusses examples of applying ML to aviation, including using real-time models to suggest actions for airplane engine alerts and developing an interpretable model to augment human decision making. The document emphasizes that success requires integrating domain expertise, complying with regulations, prioritizing safety and reliability over goals like market domination, and leveraging data efficiently rather than assuming infinite resources.

1. Dr June Andrews
Principal Data Scientist
Counter Intuitive Machine Learning
for the Industrial Internet of Things

2. First Define Intuitive Machine Learning Product Development.
… bit more of an art than a science.
Ellsworth Kelly

3. Optimize and refine products by cycling multiple times with improvements.
Machine Learning Product Development Cycle - Phases
Extending Boehm Spiral Model from Software Engineering
Plan
BuildTest
Learn
Release

4. Plan
Source
Ideas
Good companies
make good
decisions. Great
companies forgo
good decisions to
make great ones.
Proper planning
leads to efficiently
executing great
ideas.
Prioritize
Define
Success
Design
Roadmap

5. Plan
Build
Source
Ideas
Building &
Optimizing models
is largely a solved
problem.
Identify & prioritize
unsolved problems
early on.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Build &
Train
Optimize

6. Plan
BuildTest
Source
Ideas
Testing should
match the diversity
of the user base &
the gravity of the
ML Product’s
responsibility.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Build &
Train
OptimizeEvaluateDog Food

7. Plan
BuildTest
Learn
Source
Ideas
Take the time to
learn.
Small changes in
complex
environments often
reveal cascading
effects.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Build &
Train
OptimizeEvaluateDog Food
Evangelize

8. Plan
BuildTest
Learn
Release
Source
Ideas
Model life
expectancy is
increasing.
Model Robustness
should follow.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Build &
Train
OptimizeEvaluateDog Food
Evangelize
Release
100%
Retrain &
Maintain

9. Plan
BuildTest
Learn
Release
Source
Ideas
All stages should
be addressed, even
if they are skipped.
A modification to
an upstream stage
triggers changes to
all downstream
stages.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Release
100%
Retrain &
Maintain
Build &
Train
OptimizeEvaluateDog Food
Evangelize

10. Source
Ideas
Each stage may
involve input from
many roles including
users & customers,
but each stage should
have an Owner.
Who owns Prioritize
is a reflection of
{Product, Data,
Engineering, Design}-
Driven Companies.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Release
100%
Retrain &
Maintain
Build &
Train
OptimizeEvaluateDog Food
Evangelize
Product
Data Science
Data Engineering
ML Engineering
Software Engineering
Per Company

11. Plan
BuildTest
Learn
Release
Source
Ideas
Platforms will
automate many
stages & focus
efforts on stages
that yield a
competitive
advantage.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Release
100%
Retrain &
Maintain
Build &
Train
OptimizeEvaluateDog Food
Evangelize

12. What is Counter-Intuitive about Industrial Internet of Things?
…research is driven by need based innovations.
Context. Process and people are strikingly similar. Context forces changes.

13. The Industrial Internet of Things Connects Power, Engines & People
1/3 of the World’s Power is Generated by GE.
60% of Airplane Engines are made by GE.
…
Preventative
Maintenance
Failure/
Anomaly
Detection
Assistive
Diagnosis
& Treatment
Systems
Optimization

14. ML + IIoT Involves an Ongoing Negotiation to Figure Out What is Possible
Done: Real Time In Production Service Suggesting Actions on Airplane Engine Alerts

15. Aviation: History of Innovation & Excellence
Best Practices & Decision Design has Evolved over 15+ Years of Monitoring Fleets

16. New Workflow Preserves Baseline Reliability with Increasing Speed & Accuracy
Delivered an Augmented Human Interpretable Model

17. Plan
BuildTest
Learn
Release
Source
Ideas
Compliance with
regulations &
customer
agreements is a
competitive
advantage.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Release
100%
Retrain &
Maintain
Build &
Train
OptimizeEvaluateDog Food
Evangelize
IIoT Counter-Intuitive*

18. Global Domination
is not the top line
goal.
Safety, Reliability &
Efficiency are.
IIoT Counter-Intuitive*
Plan
BuildTest
Learn
Release
Source
Ideas
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Release
100%
Retrain &
Maintain
Build &
Train
OptimizeEvaluateDog Food
Evangelize

19. Plan
BuildTest
Learn
Release
Source
Ideas
Big Data, but
not infinite &
not cheap.
Take the time,
to be clever.
Prioritize
Define
Success
Design
Roadmap
Find &
ETL Data
Feature
Engineer
Productionize
A/B Test
Analyze
Product
Review
Release
100%
Retrain &
Maintain
Build &
Train
OptimizeEvaluateDog Food
Evangelize
IIoT Counter-Intuitive*

20. Machine Learning Experts Transform Domain Knowledge into Model Inputs
Aviation Experts are a Strategic Advantage in IIoT

21. Turing Test, tests a mastery of communication.
Bob Test, tests a master of communication, data synthesis & problem solving.
Challenge to AI - The Bob Test
Bob gathers input from the monitoring team, synthesizes the data & calls the airline.
Bob works with experts from the airline to determine root cause & design safe plans
of action for quick resolution.

22. 5 Months to Augment Aviation — Eta 2 Months to Augment Power
First Order Effects are When Innovations for Planes Help Planes.
Second Order Effects are When Innovations for Planes Help Power.

23. To Dive Deeper Visit the Wise.io Booth
/DrAndrews
Thank You

24. Competitive Machine Learning Requires Strategic Talent Contributions
Augmenting Fleet Monitor was an Orchestration of Aligning Talents with Needs