1) The HTM theory proposes a biologically realistic neuron and cellular layer model that recognizes hundreds of patterns and learns via growing new synapses rather than changing weights.
2) The cellular layer model learns predictive models of temporal and sensorimotor sequences.
3) The theory deduces that every cortical column determines the allocentric location of inputs on an object, allowing columns to learn complete models of objects and infer more quickly with multiple columns.
Numenta Brain Theory Discoveries of 2016/2017 by Jeff Hawkins
1. HTM Meetup
November 3, 2017
Jeff Hawkins
jhawkins@numenta.com
Numenta Brain Theory Discoveries of 2016/2017
2. 1) Reverse Engineer the Neocortex
- biologically accurate theory
- test empirically and via simulation
2) Enable technology based on cortical theory
- active open source community
- basis for Artificial General Intelligence
- IP licensing
We have made
significant advances
on the cortical theory
5. March 30, 2016
October 25, 2017
See all papers at Numenta.com/papers
3) How columns in
cortex model objects
through movement
4) Missing Ingredient!
1) Neuron model
2) How layers of
neurons in cortex
model sequences
6. Point Neuron Model
x Real neurons are not
like this!
Integrate and fire neuron: Lapicque, 1907
Perceptron: Rosenblatt 1962;
Deep learning: Rumelhart et al. 1986; LeCun et al., 2015
Artificial Neurons
7. Real and HTM neurons recognize 100’s of unique patterns.
Most recognized patterns act as predictions.
5K to 30K excitatory synapses
- 10% proximal, can cause spike
- 90% distal, cannot cause spike
Dendrites are pattern detectors
- 15 co-active, co-located synapses
has big effect
Real Neuron HTM Neuron Model
8. Learning is by Rewiring, Forming New Synapses
Not by Changing Synaptic Weights
Biology
HTM
9. Modeling a Cellular Layer
HTM Sequence Memory
A
X B
B
C
C
Y
D
Before learning
A
X B’’
B’
C’’
C’
Y’’
D’
After learning
Same columns,
but only one cell active per column.
Sequences A-B-C-D vs. X-B-C-Y
10. March 30, 2016
October 25, 2016
See all papers at Numenta.com/papers
3) How columns in
cortex model objects
through movement
4) Missing Ingredient!
1) Neuron model
2) How layers of
neurons in cortex
model sequences
11.
12. L6b
Output
Location on object
“allocentric”
L4 (input layer)
L2/3 (output layer)
L5
L6a
HTM Sensorimotor Inference Theory (single column)
1) Every column determines allocentric location of input
2) As sensor moves, column is exposed to different
feature/locations on object
3) Output layer “pools” feature/locations. Stable over movement.
4) Columns learn models of complete objects
Object
Input
Sensed Feature
45%Feature@Location
13. Output layer
“Object”
Input layer
“Feature/Location”
Location
on object
Column 1 Column 2 Column 3
Sensory
feature
HTM Sensorimotor Inference Theory (multiple columns)
Each column has partial knowledge of object.
Long range connections in output layer allow columns to vote.
Inference is much faster with multiple columns.
16. • Yale-CMU-Berkeley (YCB) Object Benchmark (Calli et al, 2017)
– Diverse set of objects designed for robotics grasping tasks
– 80 common physical objects
– Includes 78 complete high resolution 3D CAD files
Simulations: YCB Object Benchmark
17. • Virtual hand using the Unity game engine
• Inputs
– Curvature based sensor on each fingertip
– Both inputs are highly sparse binary vectors
• Network with 4096 neurons per layer per
column
• Results
• 98.7% recall accuracy (77/78 uniquely classified)
• Convergence time depends on object and sequence of sensations
Simulations: YCB Benchmark
25. • Our model predicts that sensory regions will contain cells tuned to the location
of features in an object's reference frame
• Movement dynamically modulates cell responses even in primary sensory
regions (Trotter and Celebrini, 1999; Werner-Reiss et al., 2003)
• Grid cells solve a similar problem, location of body in environment
“Border ownership cells”
(Willford & von der Heydt, 2015)
Evidence for Allocentric Location in Cortex
26. Summary
1) HTM Neuron Model
- Biologically more realistic
- Functionally more powerful
- Recognizes 100’s of unique patterns
- Most patterns put neuron into “predictive” state
- Learning is via grown of new synapses
2) HTM Cellular Layer Model
- Learns predictive models of sensory input
- Applied to temporal sequences
- Applied to sensorimotor sequences
3) Deduced Allocentric Location is Determined in Every Column
- Every column learns complete models of objects
- Multiple columns infer quickly
4) Allocentric Location Changes “everything”
- Columns and regions are far more powerful than
previously thought
- Changes how we think about hierarchy
- Progress on understanding rest of cortex will accelerate
- Implications for robotics and machine intelligence