Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ICMNS 2017
Yuwei Cui1, Yiyi Yu2, Spencer Smith2 and Subutai Ahmad1
1Numenta,Inc., Redwood City, CA
2University of North Ca...
Numenta’s Approach
NuPIC
Open source
community
Partnerships &
licenses
Academic
publications
Anomaly detection
Temporal pr...
Outline
• Cell assemblies and the HTM Model Neuron
• Network of neurons learn sequences
• Testable predictions of HTM
• Pr...
Cell assemblies
• Donald Hebb: neurons do not function in isolation but are organized
in assemblies.
• Cell assembly: “a d...
Computation of a Neuron Lacking Dendrites
Perceptron, Rosenblatt 1962; Rumelhart et al. 1986; LeCun et al., 2015
Integrate...
Active Dendrites
Non-linear
8-20 coactive synapses lead to
dendritic NMDA spikes
(Branco & Häusser, 2011; Schiller et al, ...
What Computation do Active Dendrites Perform?
Pyramidal neuron
Active dendrites exist to boost/normalize the effects of di...
“dendritic subunits” “soma”
Mel 1992; Poirazi et al., 2003; Polsky et al., 2004
Koch & Poggio 1992; Shepherd and Brayton 1...
What computation is essential in cortex?
Sequence learning
What Computation do Active Dendrites Perform?
10
HTM Neuron Models Proximal and Active Dendrites
HTM neuronPyramidal neuron
Feedforward
Linear summation
Binary activation
...
Network of Neurons with Active Dendrites
Neurons in the same column have the same feedforward connections
Different neuron...
Doubly Sparse Activation due to Two Inhibition Systems
Intercolumn inhibition allows a sparse set of columns to become act...
Feedforward Input
Sparse activation of columns
(intercolumn inhibition)
No prediction
All cells in column become active
(s...
High Order Sequences Two sequences: A-B-C-D
X-B-C-Y
Columns with depolarized cells
represent predictions
15
1) On-line learning
2) High-order representations
For example: sequences “ABCD” vs. “XBCY”
3) Fully local and unsupervised...
Outline
• Cell assemblies and the HTM Model Neuron
• Network of neurons learn sequences
• Testable predictions of HTM
• Pr...
1) Presence of cell assemblies in the cortex with natural stimulus
2) Sparser activations during a predictable sensory str...
0 1000 2000 3000 4000 5000
Time
20
40
60
80
100
Neuron
0
10
20
30
40
NMDAspike#
0.002
0.010
0.020
sparsity
HTM: Sparse Act...
• Simultaneous 2-photon calcium imaging from V1 and AL
Data
20
Stirman et al. 2016, Nature Biotechnology
Spencer Smith
Visual Stimulus
30 seconds naturalistic stimuli, 20 repeats
Yiyi, Yu
Experiment
Population response
Result 1: Increasing sparsity across repeats
23
• Sparsity: 1% → 0.4%
Result 1: Increasing sparsity across repeats
24
• Sparsity: 1% → 0.4%
• Subset Index:
# cells active on both the first and...
Result 2: Presence of high-order cell assemblies
25
• k-cell assembly: a set of k cells that are coactive together (<75 ms...
Result 3: Cell Assemblies are locked to visual stimulus
27
Result 3: Cell Assemblies are locked to visual stimulus
28
• Distribution of cell assembly occurrence times relative to th...
Summary
29
Model
• Networks of neurons with active dendrite forms a powerful sequence learning
algorithm.
• Active dendrit...
30
Spencer Smith Yiyi, Yu Subutai Ahmad
Collaborators
Contact info:
ycui@numenta.com
We invite applications for our visiti...
ICMNS Presentation: Presence of high order cell assemblies in mouse visual cortices during natural movie stimulation
Upcoming SlideShare
Loading in …5
×

ICMNS Presentation: Presence of high order cell assemblies in mouse visual cortices during natural movie stimulation

444 views

Published on

This is a presentation that Yuwei Cui delivered at the International Conference on Mathematical Neuroscience in Boulder, Colorado in June 2017.

Published in: Science
  • If u need a hand in making your writing assignments - visit ⇒ www.HelpWriting.net ⇐ for more detailed information.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Yes you are right. There are many research paper writing services available now. But almost services are fake and illegal. Only a genuine service will treat their customer with quality research papers. ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Read this fabulous site and invested in a steam shower and never glanced back, fantastic resources on this website cannot say thanks enough. HelpWriting.net
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

ICMNS Presentation: Presence of high order cell assemblies in mouse visual cortices during natural movie stimulation

  1. 1. ICMNS 2017 Yuwei Cui1, Yiyi Yu2, Spencer Smith2 and Subutai Ahmad1 1Numenta,Inc., Redwood City, CA 2University of North Carolina, Chapel Hill, NC Presence of high order cell assemblies in mouse visual cortices during natural movie stimulation 1
  2. 2. Numenta’s Approach NuPIC Open source community Partnerships & licenses Academic publications Anomaly detection Temporal prediction Natural language Technology Validation and Development Numenta Research HTM theory Experimental Neuroscience *HTM = Hierarchical Temporal Memory 1) Discover operating principles of the neocortex 2) Create technology based on those principles
  3. 3. Outline • Cell assemblies and the HTM Model Neuron • Network of neurons learn sequences • Testable predictions of HTM • Presence of cell assemblies in the mouse visual cortex
  4. 4. Cell assemblies • Donald Hebb: neurons do not function in isolation but are organized in assemblies. • Cell assembly: “a diffuse structure comprising cells in the cortex and diencephalon, capable of acting briefly as a closed system, delivering facilitation to other such systems”. - "The Organization of Behavior" (1949) How does cortex compute with cell assemblies? How do neurons form and recognize cell assemblies?
  5. 5. Computation of a Neuron Lacking Dendrites Perceptron, Rosenblatt 1962; Rumelhart et al. 1986; LeCun et al., 2015 Integrate and fire neuron, Lapicque, 1907 LNP cascade model, Chichilnisky 2001; Paninski 2004; “axon”“soma” Point Neuron Model 6
  6. 6. Active Dendrites Non-linear 8-20 coactive synapses lead to dendritic NMDA spikes (Branco & Häusser, 2011; Schiller et al, 2000; Losonczy, 2006; Antic et al, 2010; Major et al, 2013; Spruston, 2008; Milojkovic et al, 2005, etc.) Major, Larkum and Schiller 2013 7 Pyramidal neuron Cell assemblies are required to trigger dendritic spike.
  7. 7. What Computation do Active Dendrites Perform? Pyramidal neuron Active dendrites exist to boost/normalize the effects of distal synapses, leading in the end to a more linear, point-neuron-like cell. (Spencer and Kandel 1961; Shepherd et al. 1985; Cauller and Conors 1992; Bernander, Koch et al. 1994; De Schutter and Bower 1994; Cook and Johnston 1997; Magee 1998; Cash and Yuste 1999; Williams and Stuart 2000; London and Hausser 2005). Neurons with active dendrites can exhibit integrative behaviors more complex than that of a point neuron. 8
  8. 8. “dendritic subunits” “soma” Mel 1992; Poirazi et al., 2003; Polsky et al., 2004 Koch & Poggio 1992; Shepherd and Brayton 1987 Two layer neural network Pyramidal neuron What Computation do Active Dendrites Perform? 9
  9. 9. What computation is essential in cortex? Sequence learning What Computation do Active Dendrites Perform? 10
  10. 10. HTM Neuron Models Proximal and Active Dendrites HTM neuronPyramidal neuron Feedforward Linear summation Binary activation Feedforward Linear Generate spikes Basic receptive field Local Threshold coincidence detectors “Predicted” cell state Local Learn transitions between activity patterns FeedbackFeedback Invokes top-down expectations Hawkins & Ahmad, Front. Neural Circuits, 2016 11
  11. 11. Network of Neurons with Active Dendrites Neurons in the same column have the same feedforward connections Different neurons in the same column has different lateral connections Feedforward connections onto the proximal dendrites activate neurons directly Lateral connections onto the basal dendrites depolarize neurons without generating an immediate action potential Feedforward Input Active Inactive Depolarized (predicted) Lateral (contextual) Input 12
  12. 12. Doubly Sparse Activation due to Two Inhibition Systems Intercolumn inhibition allows a sparse set of columns to become active Depolarized cells will fire faster and prevent other cells in the same column from firing through intracolumn inhibition when feedforward input arrives Active Inactive Depolarized (predicted) Intercolumn inhibitionIntracolumn inhibition Feedforward Input 13
  13. 13. Feedforward Input Sparse activation of columns (intercolumn inhibition) No prediction All cells in column become active (surprise/oddball response) Arranging Neurons In Minicolumns Leads To Powerful Sequence Memory & Prediction Algorithm t-1 t Two separate sparse distributed representations No prediction A subset of cells are depolarized via predictive contextual input With prediction Feedforward Input With prediction Only predicted cells in column become active (stimulus-specific adaptation) 14
  14. 14. High Order Sequences Two sequences: A-B-C-D X-B-C-Y Columns with depolarized cells represent predictions 15
  15. 15. 1) On-line learning 2) High-order representations For example: sequences “ABCD” vs. “XBCY” 3) Fully local and unsupervised learning rules 4) Extremely robust Tolerant to >40% noise and faults (neuron death) 5) Fast adaptation to changes 6) Multiple simultaneous predictions For example: “BC” predicts both “D” and “Y” 7) High capacity HTM Sequence Memory : Computational Properties Extensively tested, deployed in commercial applications Full source code and documentation available: numenta.org & github.com/numenta Papers available: (Hawkins & Ahmad, Front. Neural Circuits, 2016; Cui et al., 2015, 2016) 16
  16. 16. Outline • Cell assemblies and the HTM Model Neuron • Network of neurons learn sequences • Testable predictions of HTM • Presence of cell assemblies in the mouse visual cortex
  17. 17. 1) Presence of cell assemblies in the cortex with natural stimulus 2) Sparser activations during a predictable sensory stream. (Vinje & Gallant, 2002) 3) Unanticipated inputs leads to a burst of activity correlated vertically within mini-columns. 4) Neighboring mini-columns will not be correlated. (Ecker et al, 2010) 5) Predicted cells need fast inhibition to inhibit nearby cells within mini-column. Plasticity & Learning: 7) Localized synaptic plasticity for dendritic segments that have spiked followed a short time later by a back action potential. (Losonczy et al, 2008) 8) The existence of localized weak LTD when an NMDA spike is not followed by an action potential. 9) The existence of sub-threshold LTP (in the absence of NMDA spikes) in dendritic segments if a cluster of synapses become active followed by a bAP. Testable Predictions 18
  18. 18. 0 1000 2000 3000 4000 5000 Time 20 40 60 80 100 Neuron 0 10 20 30 40 NMDAspike# 0.002 0.010 0.020 sparsity HTM: Sparse Activity Becomes Sparser For Learned Sequences 19 2% sparsity 0.6% sparsity More Dendritic spikes Active cells after learning is a subset of cells before learning
  19. 19. • Simultaneous 2-photon calcium imaging from V1 and AL Data 20 Stirman et al. 2016, Nature Biotechnology Spencer Smith
  20. 20. Visual Stimulus 30 seconds naturalistic stimuli, 20 repeats Yiyi, Yu
  21. 21. Experiment Population response
  22. 22. Result 1: Increasing sparsity across repeats 23 • Sparsity: 1% → 0.4%
  23. 23. Result 1: Increasing sparsity across repeats 24 • Sparsity: 1% → 0.4% • Subset Index: # cells active on both the first and the last trial # cells active on the last trial
  24. 24. Result 2: Presence of high-order cell assemblies 25 • k-cell assembly: a set of k cells that are coactive together (<75 ms) at least once. • Method: Enumerate the number of distinct k-cell assemblies (352x8520) • Cell assemblies occurs more frequently than predicted by the surrogate models, the difference is larger for higher-order patterns Poisson model assumes each neuron fires with a homogeneous firing rate Shuffled model shuffles spikes across repeats for individual neurons, it preserves all the stimulus driven responses, but destroys non-stimulus driven correlations.
  25. 25. Result 3: Cell Assemblies are locked to visual stimulus 27
  26. 26. Result 3: Cell Assemblies are locked to visual stimulus 28 • Distribution of cell assembly occurrence times relative to the median occurrence time
  27. 27. Summary 29 Model • Networks of neurons with active dendrite forms a powerful sequence learning algorithm. • Active dendrite requires repeatable cell assemblies in a neural population. Data • Repeatable cell assemblies are found in mouse visual cortex during natural movie stimulation. • Cell assemblies cannot be explained by stimulus response properties of individual neurons.
  28. 28. 30 Spencer Smith Yiyi, Yu Subutai Ahmad Collaborators Contact info: ycui@numenta.com We invite applications for our visiting scholar program scholars@numenta.com https://numenta.com/careers-and-team/careers/visiting-scholar-program/ Jeff Hawkins

×