Story line:
By now you have heard a lot about the promise and the limits of CG. This presentation is meant to show you how we can realize more of the promise by overcoming some important limitations. The way I’ll demonstrate this is by showing how two core ideas that have been around for a while when implemented well can increase the power of machines to act like super brains. The two ideas are AM and CD based on KC.
• Love story between Kolmogorov Complexity (KC) & Associative Memories (AM).
Associative memory means the ability to associate huge amounts of data and find pattern in real time much faster than human beings can do. At Saffron Technologies we have scaled AM to Big Data. We use CD a way to find meaning in the data and reason likes humans do but much more powerfully and faster. When CD and AM are combined it is like a match made in heaven that realizes the promise of cognitive computing. This perfect match overcomes the inability of most machine learning approaches to be able to add new knowledge on the fly in a consistent way.
• Use cases.
I’m going to share with you 2 applications where we have applied cognitive computing. Where we are helping human beings to make decisions based on data that had already existed but had no meaning until we applied AM.
o Gates
o Mount Sinai
• Why are KC & AM a match made in heaven?
What draws our lovers AM & CD together are 3 qualities.
o Universality
o Context matters
o Compression – sparse coding
• What is Kolmogorov Complexity?
o Discern the signal from the noise to make better decisions
o Alice and the judge
• How do use KC ⊕ as absolute measure for information distance between objects
o Cowboy, saddle and movie
• We need context to resolve ambiguity
o Meaning comes from context
o Cognitive distance allows for context
o Associative Memories allow for context since they implement graph
• What are associative memories?
One message, the weights are deterministic. The weights are the strength of the connection between the neurons. These weights are deterministic. We do not need optimization to calculate them but they are baked in.
• How has Saffron Technologies implemented them?
• What are the applications of Cognitive Distance on top of Associative Memories?
• Summary – What is Saffron’s contribution to cognitive computing?
3. Early Warning System
Protect The Foundation from physical and reputation threats
Detect weak signals to predict threat
Early warning system to score threats from people & groups based
on dynamic incremental machine learning
Structured and Unstructured Data
Incidence Reporting
Metadata + E-mails
Harvested Web Pages
(Terabytes & growing )
Strategic Early Warning
System – Igor Ansoff
Scan environment to
detect weak signals &
rare events to predict
surprises
4. Pattern Recognition In Healthcare
Automate Echocardiogram Diagnoses
90 metrics, 6 locations, 20 time frames
10,000 attributes/beat*patient
-> 100 million triples / beat*patient
Heat maps show separation of disease
states. Associations between variables in
restrictive cardiomyopathy (red) separate
from dominant associations in constrictive
pericarditis (green)
Intelligent Platforms for Disease Assessment
Novel Approaches in Functional Echocardiograph,
Partho P. Sengupta, in JACC: Cardiovascular Imaging, 11/2013
Saffron 90%
Best doctor 76%
State of the art
C-tree 54% using 7
attributes
5. Watch The Video With Dr. Sengupta
Part 1
http://www.youtube.com/watch?v=rGkyDkDmZts
10:30 - nice Big data setup
12:30 - 14:00 Intelligent Computing
Part 2
http://www.youtube.com/watch?v=SAby6-tMvng
4:40 - Look inside the dataset as a matrix
5:30 - Saffron <<< here it is
6:16 - Associate Memory Reasoning
7:17 - heat map where I can see a pattern
7:56 - 8:26 compare patterns and accuracy of 89.6%
8:51 - 9:07 need to do pattern recognition for intelligent assessment
11/22/20
13
5
Saffron Technology, Inc. All Rights Reserved.
6. Match Made in Heaven
Cognitive Distance Associative Memories
Universality
• Cognitive Distance is universal
•
C. Bennett, IBM, 1997; M Hutter, IDSIA, 2000 AIXI
• Nonparametric, incremental, deterministic weights
Context
• Cognitive Distance depends on context
• AM fabric stores context – complete graph
Compression
• K Complexity measures compressibility
• Associative Memories are perfect compressor
7. Kolmogorov Complexity – Signal vs. Noise
Snake eyes are regular sequence -> regular cause, meaning
probability > 0
for snake eyes!
100X
Place a huge bet on
simple outcomes – fair
dice have no pattern
8. How Do Extract Similarity Automatically?
xy=73M
What is closer to cowboy?
1. saddle or
2. movie
“movie”
y=1,890M
xy=8M
x=131M
Cognitive Distance based on Kolmogorov Complexity
Approximating Kolmogorov Complexity K(x) ~ log x/N we get
CD ~ max {log(fx),log(y)}-log(x,y) / ( logN-min{log(x),log(y)}
the saddle is closer to the cowboy
“saddle”
y=87M
9. Not Always So Easy - Context Resolves Ambiguity
Cognition Is About Context
Cognitive Distance Allows for Condition
CD|c ~ max {log(xc|c),log(yc|c)}-log(xc,yc|c) /
( logN-min{log(xc|c),log(yc|c)} )
11. NoSQL - Associative Memories Are Truly
Asynchronous Computing
Ising Model for order disorder phase transition
e.g. Ferromagnetism
H = -J / 2å SiSj - hå Si
i, j
i
Hopfield Network
weights are
deterministic
parameter free
Connections and counts
synapses and strengths
12. Saffron’s Solution - Large Scale Machine Learning on
Sparse Matrices
Build the Brain
1. Unify structured & un-structured data
2. Extract entities
3. Build semantic graph with counts on edges stored as triples
John Smith flew to London on 14 Jan 2009 aboard United Airlines to meet with Prime Minister for 2 hours on a rainy day.
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
verb
flew 1
1
1
1
1
1
1
1
1
verb
meet 1
1
1
1
1
1
1
1
1
keyword
rainy 1
1
1
1
1
1
1
1
1
keyword
day 1
1
1
1
1
1
1
1
keyword
aboard 1
1
1
1
1
1
1
1
1
duration
2 hours 1
1
1
1
1
1
1
1
1
1
1
rainy 1
1
1
1
1
1
1
keyword
day 1
1
1
1
1
1
1
1
keyword
aboard 1
1
1
1
1
1
1
1
1
duration
2 hours 1
1
1
1
1
1
1
1
1
1
keyword
keyword
verb
verb
time
person
place
person
1
day
1
1
1
& 1
&
1
1
1
1
1
1
1
United&
Airlines& 1
1
1
& 1
&
1
1
1
1
1
1
14<Jan<
09 1
1
1
1
& 1
&
1
1
1
1
1
verb&
flew& 1
1
1
1
1
& 1
&
1
1
1
1
verb&
meet& 1
1
1
1
1
1
& 1
&
1
1
1
keyword&
rainy& 1
1
1
1
1
1
1
& 1
&
1
1
keyword&
day& 1
1
1
1
1
1
1
1
& 1
&
1
keyword&
aboard& 1
1
1
1
1
1
1
1
1
& 1
&
duration
2&
hours& 1
1
1
1
1
1
1
1
1
1
day&
aboard&
2&
hours&
duration
1234
refid&
Place&&&&&&
&&&&&&
&&&&&
London
&
ketword&
organization&
keyword&
Prime&
Minster& 1
time
person&
flew&
1
1
rainy&
1
1
meet&
1
1
verb&
1
1
keyword&
1
1
14<Jan<09
1
1
verb&
1
1
United&
&
Airlines
1
John&
Smith 1
organization&
1
& 1
&
person&
John&
Smith
1
Person
Prime Minister
Prime&
&
Minster
1234
aboard
1
ketword
1
flew
meet 1
rainy
verb
meet
1
14-Jan-09
1
Prime Minster
1
London
1
John Smith
flew 1
1
1
day
1
verb
2 hours
1
duration
1
rainy
1
1
aboard
1
14-Jan-09 1
ketword
United Airlines 1
time
keyword
organization
1
keyword
1
1
flew
1
1
meet
1
1
verb
1
1
14-Jan-09
1
1
1
verb
1
1
John Smith
1
1
1
United Airlines
1
time
John Smith 1
person
London 1
person
organization
place
1
1234
1
1
1
1
1
London
1
1
1
1
1
1
refid
1
1
1
1
1
1
place
1
1
1
1
1
1
1
1234
1
1
1
1
1
14-Jan-09 1
refid
1
1
1
1
1
Prime Minister 1
time
1
1
1
1
time
refid&
1
1
1
person
Organization
United Airlines
1234
1
1
keyword
Why is this so special?
• Non-parametric, nonlinear & instant
incremental learning
• Graph & statistics
• Millions of features
• Saffron stores &
queries billions of triple
counts
refid
1
1
2 hours
1
duration
1234
London 1
John Smith 1
person&
refid
place
person
person
Make the Brain Think
• Reason by similarity with
cognitive distance
13. Happy Ending – Offspring of KC & AM
Discovery – Search
– Entity ranking and semantic context
– Convergence – the distance over time
Classification
– Predicting risk (bad, good)
– Customer life time value
– Echocardiogram diagnosis
Clustering
– Evolutionary trees, languages, music
– Novelty detection: spare parts, planes, etc.
another example of predicting at the personal (consumer) level
3 univcomp machines: von Neumann architecture – CPU and RAM; cellular automata (von Neum & StaniUlam); associative memories –synapses as compute and storage unit -> content addressable associative memory -> asynchronous, reaching fixed point - Hopfield nets (homomorph to Ising model -> node's behavior is deterministic moves to a state to minimize energy of itself & its neighbors -> Lapunov, emerging patterns