How AI can improve decision-making in organizations by leveraging knowledge and opinions of experts.

1. September 2019
Human Collective Intelligence
The Future of Corporate Innovation

2. 1
Transforming Wisdom of Customer Crowds: The Challenge
1999: Lego Direct was created to be a global
direct-to-consumer “corporate startup”
encompassing LEGO.com, LEGO Shop at Home
and other LEGO entities
2001: LEGO creates first consumer Dialogue
Portal
2002: Message boards are co-developed
Objectives
§ Listen and respond to customers
§ Understand the consumer mindset to
guide development
§ Build and improve brand loyalty
§ Make consumer part of the company
§ Find the relationship margin and not
just the profit margin
Result: More than 250K registered users
in the first year

3. 2
Transforming Wisdom of Customer Crowds
Question to Customers: “What would you like to experience when …”
Top Ranked Response: “A long engaging experience of building”
Product Idea: The LEGO Star Wars Imperial Star Destroyer - The
largest and most expensive LEGO set ever offered.
Initial Internal Reaction: “You’ll never sell something like that.”
Results: Most profitable product launch in company history, gross
margin dollars >10x company average

4. 3
Learning from Conversations with Customers: 2001 - 2008

5. 4
Learnings from Innovating with Collective Intelligence
§ Rating as listening company up from 18% to 72%
§ Uncover unmet market needs to generate product ideas
§ Reduced time to market
§ Cost-effective product testing
§ Solicited targeted, rapid insights on key shows
§ Uncovered shifting attitudes over days/weeks
§ Made changes before viewers changed behaviors
Engaging customers as partners in innovation
creates radical positive change

6. 5
A New Generation of Human-First AI
§ Move beyond feedback management and knowledge
models
§ Acquire knowledge through assessment strategies
§ Prove its power: started with predicting startup success
§ Integrate the best of knowledge system and AI
technologies

7. 6
Three Foundational Technologies Required
§ Computational models of expertise (Symbolic AI – first
wave)
§ Automation of knowledge acquisition via collective
intelligence
§ Bayesian Inference that learns from data

8. 7
What We Do
CrowdSmart is a pioneer in the use of human-
first artificial intelligence designed to
improve decision-making in organizations, by
leveraging the diverse knowledge and opinionsof
a community of experts.

9. 8
CrowdSmart Human-Powered AI Overview
CI Learning
Algorithm + NLP
Bayesian
Belief Network
Provides transparency
and thematic
explanations for
startup scoring
Expert Input
Experts assess startups
guided by the
knowledge acquisition
system
CrowdSmart
Knowledge Model
Bayesian belief network is
influenced by 4+ years of
assessing and measuring
startups
Bayesian
Classifier
Platform Outputs
› Decision: Invest/Pass
› Probability of success
› Breakdown of decision
factors
› Expert influence ranking

10. 9
The Future of Corporate Innovation
§ Strategic investment in startups and new initiatives with
accuracy and precision
§ Collaborative innovation with customer and expert
communities to create transformative new offerings
§ Identification of thought leaders from within
§ Create a cycle of accumulating accurate learning

11. 10
Collective Prediction of Seed Stage Startup
Scores or ratings with supporting reasons allow for
predictions with explanations

12. 11
Breaking Out of Your Own Rut
§ Reach beyond the old way of doing things
§ Leverage scientific and technology-based approach to
expanding options
§ Create a system for learning fast
§ Integrate internal team views with external views

13. 12
Why Modelling the Collective Mind Matters
§ Identify precise implications of an investment
§ Understand the facts, knowledge and influencers behind a
decision or investment
§ Predict the results of the model and act
§ Create a cycle of accumulating accurate learning

14. 13
Building the Future on Human-First AI
§ Discover new market opportunities
§ Leverage the best minds to refine and predict outcomes
§ Rapidly learn your next move

15. 14
Appendix (The Science)

16. Diversity of expert opinion is the key to increasing prediction accuracy
15
Team collective prediction error = Average individual error – Prediction diversity
Two proven collective intelligence prediction accuracy facts
Collective Intelligence Science
Diversity of perspective, experience, education, and
cultural background of evaluators reduces collective
team prediction error
The collective team prediction error will always be less
than the average individual error provided the team has
prediction diversity1 2
The math of
accurate
predictions

17. 16
Essential Technology for Collective Intelligence
1. Discover new ideas from open ended question
2. Allow for evolution of ideas from group
interactions
3. Learn a statistically significant ranking
1. Rapidly learns priorities
2. Process must be stable and reproducible
3. Correctly give high rankings to important
ideas
4. Correctly give low rankings to unimportant
ideas
Requirements
Results

18. Bayesian Learning Basics
The science and math for generating accurate individual predictions
17
Bayesian Principle Ex: Evaluator Evaluation Belief Mathematical Formula
Beliefs are expressed as a
probability distribution
I believe this startup has only moderate
odds of an ROI
p(H) = Prior odds
Given evidence, how likely
is it to change beliefs
Feedback from expert team and founder
interactions are compelling
p(E|H) = Likelihood (odds change)
Update beliefs based on
evidence
I am convinced that this startup has a high
odds of an ROI
p(H|E) = Posterior odds
p(D|H)= p(H|D)p(D)
p(H)