Main Project Presentation - Computer Science Department, College of Engineering Chengannur 2003-2007, Affiliated to Cochin University of Science and Technology, Kerala, India

1. ANN Based Object Recognizing Robot College of Engineering Chengannur
ANN BASED OBJECT
RECOGNIZING ROBOT
Ann Mathew
Beena Sara Samuel
Jaison Abey Sabu
Ragesh A R

2. ABSTRACT
The aim of our project is to create an Intelligent
Robotic Control Software that observes a work area
and retrieves the object requested by the user.

3. HARDWARE REQUIREMENTS
• Robotic Arm (SCORBOT-ER 4u)
• Controller for Robotic Arm
• Web Camera with mounting facility
• Voltage Mapping Circuit
• Computer

4. SOFTWARE REQUIREMENTS
• MATLAB 7.0
– Image Acquisition Toolbox
– Data Acquisition Toolbox
– Image Processing Toolbox
– Neural Network Toolbox
– GUIDE
• SCORBASE 4.9
– Control software for SCORBOT-ER 4u
Robotic Arm

5. OUTLINE
• Image Acquisition
• Image Processing
• ANN
• Parallel Port Programming
• Circuit to convert +ve voltage to –ve
• Location calculation Algorithm
• Interrupt control algorithm
• GUI

6. ANN Based Object Recognizing Robot College of Engineering Chengannur
IMAGE ACQUISITION

7. OVERVIEW
• Web camera mounted on stand
• Top view
• MATLAB Image Acquisition Toolbox
• Image Acquired using getsnapshot
command
• Saved in an array for processing

8. ACQUIRED IMAGE

9. PROCESSING STAGES
• Convert to gray scale
• Crop the image
• Adjust Contrast
• Convert to binary image
• Sobel Edge Detection Algorithm
• Image Dilation
• Fill closed spaces (create blobs)
• Find bounding box of each object
• Create inputs for ANN

10. IMAGE PROCESSING
Acquired Image

11. IMAGE PROCESSING
Convert to Gray Scale

12. IMAGE PROCESSING
Crop Image

13. IMAGE PROCESSING
Contrast Adjust

14. IMAGE PROCESSING
Convert to Binary Image

15. IMAGE PROCESSING
Edge Detection

16. IMAGE PROCESSING
Image Dilation

17. IMAGE PROCESSING
Fill Holes

18. IMAGE PROCESSING
Find Bounding Box of each Object

19. IMAGE PROCESSING
Input to ANN

20. SOBEL EDGE DETECTION
• Performs a 2-D spatial gradient
measurement on an image.
• Uses a pair of 3x3 convolution masks, one
estimating the gradient in the x-direction
(columns) and the other estimating the
gradient in the y-direction (rows).
• A convolution mask is usually much
smaller than the actual image and is slid
over the image, manipulating a square of
pixels at a time.

21. SOBEL MASKS

22. SOBEL PROCEDURE

23. ANN Based Object Recognizing Robot College of Engineering Chengannur
ARTIFICIAL NEURAL
NETWORKS

24. INTRODUCTION
• An information-processing system that has
certain performance characteristics in common
with biological neural networks.
• Information processing occurs at many simple
elements called neurons.
• Each connection link has an associated weight,
which, in an ANN multiplies the signal
transmitted.
• Each neuron applies an activation function
(usually nonlinear) to its net input (sum of
weighted signals) to determine its output signal.

25. NEURON MODEL
∑
f
a y
x1
x
x
2
N
w
w
w
1
2
N
a w xi i
i
N
=
=
∑1

26. BASIC OPERATION
• Region within bounding box of Binary
Image is passed into the ANN
• All objects in view are passed as input to
the ANN till a match to target object is
found
• If none matches then ‘not found’ is
displayed
• If a match is found location is calculated
and passed to the arm

27. STRUCTURE
• Three layer Network is used
• No. of pixels within the bounding box of
largest object determines the no. of input
neurons
• Current implementation has 1763 input
neurons (43 x 41)
• Number of hidden layer neurons is 10
• Number of output layer neurons is 5

28. STRUCTURE
• Initial implementation had 3 neurons
• Failure to train correctly due to similarity
between pentagon and square in binary
image
• Goal = .00005
• Learning Rate = .05

29. TRAINING PATTERNS
• CIRCLE : 1 0 1 0 1
• SQUARE : 0 0 0 0 0
• PENTAGON : 1 1 1 1 1

30. DIAGRAM

31. TRANSFER FUNCTIONS
Hidden Layer Output Layer

32. TRAINING ALGORITHM
• The Back propagation Training Algorithm
is used to train the Network
• ALGORITHM
– Forward Pass
• For each neuron in the hidden layer
– Output OutH = f ( ∑input x weighth)
• For each neuron in the output layer
– Output Out = f ( ∑Outh x weighto)

33. TRAINING ALGORITHM
– Reverse Pass (Weight Updation)
• For each neuron in the output layer
– Error E = Out (1-Out) (Target – Out)
– New weighto = Old weighto + η x E x OutH
• For each neuron in the hidden layer
– Error Err = OutH (1-OutH) (∑E x weighto)
– New weight h = Old weighth + η x Err x input

34. TRAINING PROCEDURE
• 50 images with three objects in view are
used to train the ANN
• They are processed and the region within
the bounding box for each object is used
to train the network
• Thus a total of 150 object images are used
for training, using corresponding target
outputs.
• A button for training the network is
provided in the GUI

35. TRAINING IN MATLAB

36. ANN Based Object Recognizing Robot College of Engineering Chengannur
LOCATION CALCULATION

37. INTRODUCTION
• If a match to the target object is found
then the location of that object must be
determined
• The work space is divided into a 6 x 6
matrix and an inner 5 x 5 matrix. Each of
these form a 30 x 30 pixel matrix on
screen.
• Each of them is assigned a number from 1
to 61

38. GRID NUMBERS

39. Locating a Pentagon

40. PASSING THE GRID NUMBER TO
THE ARM CONTROLLER
• The grid number is passed to the arm
controller using the parallel port (8 data
lines)
• The arm controller has 8 interrupt pins to
implement external control
• Thus the grid number is converted to
some suitable 8 bit code

41. SCORBASE SOFTWARE
• The SCORBASE software can perform
operations on receiving control interrupts
• But the software does not permit logical
operators. It only provides operations of
the form
– If interrupt n is on jump to location
• Thus we developed the following code for
the 61 positions

42. 8 BIT GRID CODE
GRID POSITION CODE
1 0 0 0 0 1 0 0 0
2 0 0 0 0 0 1 0 0
3 0 0 0 0 0 0 1 0
4 0 0 0 0 0 0 0 1
5 0 0 0 1 1 0 0 0
6 0 0 0 1 0 1 0 0
- --------------------

43. 8 BIT GRID CODE
GRID POSITION CODE
- --------------------
- --------------------
- --------------------
59 1 1 1 0 0 0 1 0
60 1 1 1 0 0 0 0 1
61 1 1 1 1 1 0 0 0

44. PASSING THE LOCATION
• The 8 bit code is passed to the controller
via the parallel port.
• SCORBASE program runs in an infinite
loop waiting for an interrupt combination.
• IMPLEMENTATION ISSUE
– Positive voltage from parallel port
– Negative voltage required as interrupt

45. ANN Based Object Recognizing Robot College of Engineering Chengannur
VOLTAGE MAPPING CIRCUIT

46. MEASURED VALUES
• From Parallel Port
– High : +3.3V
– Low : +0.76 V
• Required by the controller
– Interrupt enable : -3V to -12 V
– Interrupt disable: -2V to 12 V
• Solution : Voltage mapping circuit

47. CIRCUIT DESIGN
• The circuit to convert positive voltage to
negative voltage was designed using the
IC 741 Operational Amplifier (OpAmp)
• The OpAmp is made to work as a
comparator to check if it is high or low
• 1.5 V reference voltage was used

48. CIRCUIT DIAGRAM

49. CIRCUIT

50. ANN Based Object Recognizing Robot College of Engineering Chengannur
GUI

51. SCREENSHOT

52. ANN Based Object Recognizing Robot College of Engineering Chengannur
ROBOTIC ARM

53. SPECIFICATION
• SCORBOT-ER 4u
• 5 axes + gripper
Axis 1: Base rotation
Axis 2: Shoulder
rotation
Axis 3: Elbow rotation
Axis 4: Wrist pitch
Axis 5: Wrist roll
• Max Payload : 1 kg
• Servo motors
• Feedback

54. ROBOTIC ARM IN ACTION

55. ANN Based Object Recognizing Robot College of Engineering Chengannur
CONCLUSION

56. SUMMARY OF OPERATION

57. APPLICATION
• Waste Screening on Conveyor Belt
• Egg screening
• Basic Concept of
Observe – Identify – Locate – Operate
can be used for several Robotic Applications
– Eg. Medical Application

58. FUTURE SCOPE
• The ANN can be trained for complex color
images
• Manual control over the axis can be
obtained by using the appropriate DLL file.
• Other precision applications can be
developed

59. ANN Based Object Recognizing Robot College of Engineering Chengannur
THANK YOU