Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Neural networks (under graduate course) Lecture 1 of 9


Published on

Undergraduate course content:
Introduction and a historical review
Neural network concepts
Basic models of ANN
Linearly separable functions
Non Linearly separable functions
NN Learning techniques
Associative networks
Mapping networks
Spatiotemporal Network
Stochastic Networks

Published in: Education
  • Login to see the comments

Introduction to Neural networks (under graduate course) Lecture 1 of 9

  1. 1. Neural Networks Dr. Randa Elanwar Lecture 1
  2. 2. Lecture Content • Introduction and a historical review: – History of neurocomputing. – Overview of neurocomputing. 2Neural Networks Dr. Randa Elanwar
  3. 3. History of Neurocomputing • Alexander Bain (1873) claimed that both thoughts and body activity resulted from interactions among neurons within the brain. • For Bain, every activity led to the firing of a certain set of neurons. When activities were repeated, the connections between those neurons strengthened. According to his theory, this repetition was what led to the formation of memory. • The general scientific community at the time was doubting Bain’s theory because it required what appeared to be an excessively large number of neural connections within the brain. 3Neural Networks Dr. Randa Elanwar
  4. 4. History of Neurocomputing • It is now apparent that the brain is exceedingly complex and that the same brain “wiring” can handle multiple problems and inputs. • Neural network theory has served both to: – better identify how the neurons in the brain function and – provide the basis for efforts to create artificial intelligence. 4Neural Networks Dr. Randa Elanwar
  5. 5. Overview of Neurocomputing • The brain is said to be composed of natural neural network, I.e., mass of highly non-linear parallel inter-connected computational units called neurons. • Each neuron is connected to many other neurons • Neurons transmit signals to each other • Whether a signal is sent, depends on the strength of the bond between two neurons • Whether a signal is transmitted is an all-or-nothing event (the electrical potential in the cell body of the neuron is thresholded: fired up/ activated) 5Neural Networks Dr. Randa Elanwar
  6. 6. Brain vs. Digital Computers - Computers require hundreds of cycles to simulate a firing of a neuron. - The brain can fire all the neurons in a single step. Parallelism - Serial computers require billions of cycles to perform some tasks but the brain takes less than a second. e.g. Face Recognition 6Neural Networks Dr. Randa Elanwar Overview of Neurocomputing
  7. 7. 7Neural Networks Dr. Randa Elanwar Overview of Neurocomputing
  8. 8. Neural Networks Dr. Randa Elanwar 8 Overview of Neurocomputing What are Neural Networks • Neural Networks (NNs) are networks of neurons, for example, as found in real (i.e. biological) brains. • Artificial Neurons are crude approximations of the neurons found in brains. They may be physical devices, or purely mathematical constructs. • Artificial Neural Networks (ANNs) are networks of Artificial Neurons, and hence constitute crude approximations to parts of real brains. They may be physical devices, or simulated on conventional computers.
  9. 9. Neural Networks Dr. Randa Elanwar 9 Overview of Neurocomputing What are Neural Networks • From a practical point of view, an ANN is just a parallel computational system, consisting of many simple processing elements, connected together in a specific way in order to perform a particular task, which is difficult to traditional (serial) computers. • One should never lose sight of how crude the approximations are, and how over-simplified our ANNs are compared to real brains.
  10. 10. Comparison between brain verses ANN Neural Networks Dr. Randa Elanwar 10 Brain ANN Speed Few ms. Few nano sec. massive parallel processing Size and complexity 1011 neurons & 1015 interconnections Depends on designer Storage capacity Stores information in its interconnection (synapse) No Loss of memory Contiguous memory locations loss of memory may happen sometimes. Tolerance Has fault tolerance Limited fault tolerance: Information gets disrupted when interconnections are disconnected Control mechanism Complicated involves chemicals in biological neuron Simpler in ANN
  11. 11. Neural Networks Dr. Randa Elanwar 11 NON-LINEARITY It can model non-linear systems INPUT-OUTPUT MAPPING It can derive a relationship between a set of input & output responses ADAPTIVITY The ability to learn allows the network to adapt to changes in the surrounding environment EVIDENTIAL RESPONSE It can provide a confidence level to a given solution Advantages of NN
  12. 12. Neural Networks Dr. Randa Elanwar 12 CONTEXTUAL INFORMATION Knowledge is presented by the structure of the network. Every neuron in the network is potentially affected by the global activity of all other neurons in the network. Consequently, contextual information is dealt with naturally in the network. FAULT TOLERANCE Distributed nature of the NN gives it fault tolerant capabilities NEUROBIOLOGY ANALOGY Models the architecture of the brain Advantages of NN
  13. 13. Who uses NN 13Neural Networks Dr. Randa Elanwar
  14. 14. Neuro products and application areas 14Neural Networks Dr. Randa Elanwar
  15. 15. Neural Networks Dr. Randa Elanwar 15 How does the nervous system works? • The human nervous system can be broken down into three stages that may be represented in block diagram form as: – The receptors collect information from the environment – e.g. photons on the retina. – The effectors generate interactions with the environment – e.g. activate muscles. – The flow of information/activation is represented by arrows – feed forward and feedback. Overview of Neurocomputing
  16. 16. Overview of Neurocomputing The structure of a biological neuron Neural Networks Dr. Randa Elanwar 16 •A biological neuron has three types of main components; dendrites, soma (or cell body) and axon. •Dendrites receives signals from other neurons. •The soma, sums the incoming signals. When sufficient input is received, the cell fires; that is it transmit a signal over its axon to other cells.
  17. 17. 17 Neural network: Definition • Neural network: information processing paradigm inspired by biological nervous systems, such as our brain • Structure: large number of highly interconnected processing elements (neurons) working together • Like people, they learn from experience (by example) Neural Networks Dr. Randa Elanwar
  18. 18. Artificial Neural Network: Definition • The idea of ANN: NNs learn relationship between cause and effect or organize large volumes of data into orderly and informative patterns. • Definition of ANN: “Data processing system consisting of a large number of simple, highly interconnected processing elements (artificial neurons) in an architecture inspired by the structure of the cerebral cortex of the brain” (Tsoukalas & Uhrig, 1997). 18Neural Networks Dr. Randa Elanwar
  19. 19. Why Study Artificial Neural Networks? • Artificial Neural Networks are powerful computational systems consisting of many simple processing elements connected together to perform tasks analogously to biological brains. • They are massively parallel, which makes them efficient, robust, fault tolerant and noise tolerant. • They can learn from training data and generalize to new situations. • They are useful for brain modeling and real world applications involving pattern recognition, function approximation, prediction, … 19Neural Networks Dr. Randa Elanwar
  20. 20. What are Artificial Neural Networks Used for? • Brain modeling – Models of human development – help children with developmental problems – Simulations of adult performance – aid our understanding of how the brain works – Neuropsychological models – suggest remedial actions for brain damaged patients • Real world applications – Financial modeling – predicting stocks, shares, currency exchange rates – Other time series prediction – climate, weather, airline marketing tactician – Computer games – intelligent agents, backgammon, first person shooters – Control systems – autonomous adaptable robots, microwave controllers – Pattern recognition – speech recognition, hand-writing recognition, sonar signals – Data analysis – data compression, data mining – Noise reduction – function approximation, ECG noise reduction – Bioinformatics – protein secondary structure, DNA sequencing 20Neural Networks Dr. Randa Elanwar