An introduction to computer vision in Python, from the general concept to its implementation with some current open-source libraries. Demonstrates a selection of basic computer vision examples using SciPy, OpenCV and Pygame.

1. Motivation & Background
Computer Vision in Python
More Information
Summary
Introduction to using Python in Computer Vision
Kiwi PyCon, Christchurch, 2009
Brian Thorne
University of Canterbury
6th November 2009
Brian Thorne Computer Vision in Python

2. Motivation & Background
Computer Vision in Python
More Information
Summary
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

3. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

4. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Vision
25% of the whole brain is for vision. Around 50% of cerebral
cortex is for vision, 80% of the brain is associated with vision
in some manner.
Brian Thorne Computer Vision in Python

5. Computer Vision
Definition
The goal of computer vision is to recognize objects and their motion
What is it used for?
Scene reconstruction
Event detection
Video tracking
Object recognition
Learning
Indexing
Motion estimation
Image restoration

6. Computer Vision
Definition
The goal of computer vision is to recognize objects and their motion
What is it used for?
Scene reconstruction
Event detection
Video tracking
Object recognition
Learning
Indexing
Motion estimation
Image restoration

7. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Computer Vision crosses over with many domains
Brian Thorne Computer Vision in Python

8. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
What makes it hard?
What we see What the computer sees
Brian Thorne Computer Vision in Python

9. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Vision is inferential
http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html
Brian Thorne Computer Vision in Python

10. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

11. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Visual Object Classes Challenge 09
http://www.pascal-network.org/challenges/VOC/voc2009
Brian Thorne Computer Vision in Python

12. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Object Recognition and Segmentation - Texture
−− − −
− − −→
(Sharon, Balun, Brandt, Basri)
Brian Thorne Computer Vision in Python

13. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Object Recognition and Segmentation - Edges
http://www.robots.ox.ac.uk/~vdg/dynamics.html
Brian Thorne Computer Vision in Python

14. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Traffic Monitoring
Brian Thorne Computer Vision in Python

15. Motivation & Background
Computer Vision in Python Computer Vision
More Information Uses & Examples
Summary
Augented Reality - Sixth Sense
’SixthSense’ is a wearable gestural interface that augments the
physical world around us with digital information and lets us use
natural hand gestures to interact with that information.
http://www.pranavmistry.com/projects/sixthsense/
Brian Thorne Computer Vision in Python

16. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

17. Python In Computer Vision: OpenCV
Provides well tested, optimized and
open source code for image processing
and computer vision
Written in C, ensuring both fast and
portable.
Has been compiled for many
embedded platforms
Has multiple language wrappers
including 3 for Python
Tools have been made to use graphics
hardware to accelerate CV
performance on the GPU
Project home page and documentation is at:
http://opencv.willowgarage.com

18. Python In Computer Vision: OpenCV
Provides well tested, optimized and
open source code for image processing
and computer vision
Written in C, ensuring both fast and
portable.
Has been compiled for many
embedded platforms
Has multiple language wrappers
including 3 for Python
Tools have been made to use graphics
hardware to accelerate CV
performance on the GPU
Project home page and documentation is at:
http://opencv.willowgarage.com

19. Python In Computer Vision: OpenCV
Provides well tested, optimized and
open source code for image processing
and computer vision
Written in C, ensuring both fast and
portable.
Has been compiled for many
embedded platforms
Has multiple language wrappers
including 3 for Python
Tools have been made to use graphics
hardware to accelerate CV
performance on the GPU
Project home page and documentation is at:
http://opencv.willowgarage.com

20. Python In Computer Vision: OpenCV
Provides well tested, optimized and
open source code for image processing
and computer vision
Written in C, ensuring both fast and
portable.
Has been compiled for many
embedded platforms
Has multiple language wrappers
including 3 for Python
Tools have been made to use graphics
hardware to accelerate CV
performance on the GPU
Project home page and documentation is at:
http://opencv.willowgarage.com

21. Python In Computer Vision: OpenCV
Provides well tested, optimized and
open source code for image processing
and computer vision
Written in C, ensuring both fast and
portable.
Has been compiled for many
embedded platforms
Has multiple language wrappers
including 3 for Python
Tools have been made to use graphics
hardware to accelerate CV
performance on the GPU
Project home page and documentation is at:
http://opencv.willowgarage.com

22. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensional
arrays to Python
Well used and tested libraries for
scientific computing
Includes lots of handy tools such as
optimisation and signal processing
used often in computer vision.
Usually used with iPython and
matplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python

23. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensional
arrays to Python
Well used and tested libraries for
scientific computing
Includes lots of handy tools such as
optimisation and signal processing
used often in computer vision.
Usually used with iPython and
matplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python

24. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensional
arrays to Python
Well used and tested libraries for
scientific computing
Includes lots of handy tools such as
optimisation and signal processing
used often in computer vision.
Usually used with iPython and
matplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python

25. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensional
arrays to Python
Well used and tested libraries for
scientific computing
Includes lots of handy tools such as
optimisation and signal processing
used often in computer vision.
Usually used with iPython and
matplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python

26. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensional
arrays to Python
Well used and tested libraries for
scientific computing
Includes lots of handy tools such as
optimisation and signal processing
used often in computer vision.
Usually used with iPython and
matplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python

27. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Pygame
Game development framework
Now has basic Computer Vision support
Being Python it can be used with other Python tools -
integrates well with numpy/scipy
pygame can be downloaded from: http://www.pygame.org
Brian Thorne Computer Vision in Python

28. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Pygame
Game development framework
Now has basic Computer Vision support
Being Python it can be used with other Python tools -
integrates well with numpy/scipy
pygame can be downloaded from: http://www.pygame.org
Brian Thorne Computer Vision in Python

29. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python In Computer Vision: Pygame
Game development framework
Now has basic Computer Vision support
Being Python it can be used with other Python tools -
integrates well with numpy/scipy
pygame can be downloaded from: http://www.pygame.org
Brian Thorne Computer Vision in Python

30. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Pycam
This is the project with all the examples for this presentation. Has
a bunch of simple examples like filtering and background
subtraction, face detection.
Contains two video player classes that can work with different
backend setups, and can incorporate optional process
functions.
Examples of intergrating OpenCV with pygame - eg for eye
and face detection.
OpenCV camera class that allows an opencv camera to be
used with pygame (No longer required in latest pygame)
VideoCapturePlayer
For the rest of this presentation, examples will use the video
capture code (with error checking) from pycam.
Brian Thorne Computer Vision in Python

31. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Pycam
This is the project with all the examples for this presentation. Has
a bunch of simple examples like filtering and background
subtraction, face detection.
Contains two video player classes that can work with different
backend setups, and can incorporate optional process
functions.
Examples of intergrating OpenCV with pygame - eg for eye
and face detection.
OpenCV camera class that allows an opencv camera to be
used with pygame (No longer required in latest pygame)
VideoCapturePlayer
For the rest of this presentation, examples will use the video
capture code (with error checking) from pycam.
Brian Thorne Computer Vision in Python

32. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Pycam
This is the project with all the examples for this presentation. Has
a bunch of simple examples like filtering and background
subtraction, face detection.
Contains two video player classes that can work with different
backend setups, and can incorporate optional process
functions.
Examples of intergrating OpenCV with pygame - eg for eye
and face detection.
OpenCV camera class that allows an opencv camera to be
used with pygame (No longer required in latest pygame)
VideoCapturePlayer
For the rest of this presentation, examples will use the video
capture code (with error checking) from pycam.
Brian Thorne Computer Vision in Python

33. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Pycam
This is the project with all the examples for this presentation. Has
a bunch of simple examples like filtering and background
subtraction, face detection.
Contains two video player classes that can work with different
backend setups, and can incorporate optional process
functions.
Examples of intergrating OpenCV with pygame - eg for eye
and face detection.
OpenCV camera class that allows an opencv camera to be
used with pygame (No longer required in latest pygame)
VideoCapturePlayer
For the rest of this presentation, examples will use the video
capture code (with error checking) from pycam.
Brian Thorne Computer Vision in Python

34. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

35. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Acquiring & Display Of An Image
Live image acquisition is such a crucial
role in the majority of CV applications.
Example getting and showing a frame
as a most basic, but necessary test
Brian Thorne Computer Vision in Python

36. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Acquiring & Display Of An Image
Live image acquisition is such a crucial
role in the majority of CV applications.
Example getting and showing a frame
as a most basic, but necessary test
Brian Thorne Computer Vision in Python

37. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Python OpenCV: Image Capture
Example
from opencv import highgui as hg
capture = hg.cvCreateCameraCapture(0)
hg.cvNamedWindow("Snapshot")
frame = hg.cvQueryFrame(capture)
hg.cvShowImage("Snapshot", frame)
hg.cvWaitKey(10000)
Brian Thorne Computer Vision in Python

38. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

39. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurring
an image
Reduce noise,
Remove artifacts
Scale an image
“cleanly”
Create motion blur -
if done in one
direction
OpenCV includes a gaussian filter among many others
(cvSmooth function)
SciPy has a multi-dimensional Gaussian filter that acts on a
NumPy array
Or you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python

40. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurring
an image
Reduce noise,
Remove artifacts
Scale an image
“cleanly”
Create motion blur -
if done in one
direction
OpenCV includes a gaussian filter among many others
(cvSmooth function)
SciPy has a multi-dimensional Gaussian filter that acts on a
NumPy array
Or you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python

41. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurring
an image
Reduce noise,
Remove artifacts
Scale an image
“cleanly”
Create motion blur -
if done in one
direction
OpenCV includes a gaussian filter among many others
(cvSmooth function)
SciPy has a multi-dimensional Gaussian filter that acts on a
NumPy array
Or you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python

42. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurring
an image
Reduce noise,
Remove artifacts
Scale an image
“cleanly”
Create motion blur -
if done in one
direction
OpenCV includes a gaussian filter among many others
(cvSmooth function)
SciPy has a multi-dimensional Gaussian filter that acts on a
NumPy array
Or you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python

43. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurring
an image
Reduce noise,
Remove artifacts
Scale an image
“cleanly”
Create motion blur -
if done in one
direction
OpenCV includes a gaussian filter among many others
(cvSmooth function)
SciPy has a multi-dimensional Gaussian filter that acts on a
NumPy array
Or you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python

44. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurring
an image
Reduce noise,
Remove artifacts
Scale an image
“cleanly”
Create motion blur -
if done in one
direction
OpenCV includes a gaussian filter among many others
(cvSmooth function)
SciPy has a multi-dimensional Gaussian filter that acts on a
NumPy array
Or you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python

45. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Applying a Gaussian Blur with OpenCV
OpenCV Gaussian Blur
from pycam import VideoCapturePlayer as VCP
from opencv import cv
def gaussianBlur(im, filterSize=43):
result = cv.cvCreateMat(im.rows, im.cols,
im.type )
cv.cvSmooth(image,result,
cv.CV_GAUSSIAN, filterSize)
return result
if __name__ == "__main__":
VCP(gaussianBlur, "Guassian Filter").main()
Brian Thorne Computer Vision in Python

46. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Applying a Gaussian Blur with Scipy
SciPy Gaussian Blur
from scipy.ndimage.filters import gaussian_filter
from pycam import OpencvVideoCapturePlayer as VCP
from misc import scipyFromOpenCV
@scipyFromOpenCV
def gaussianBlur(np_image):
result = gaussian_filter(np_image,
sigma=(4, 4, 0),
order=0, mode=’reflect’)
return result
if __name__ == "__main__":
VCP(gaussianBlur,"Scipy Guassian Blur").main()
Brian Thorne Computer Vision in Python

47. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

48. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
In video security the camera mostly looks at the same boring
background
What we are usually interested in is when objects (eg people or
vehicles) enter or exit a scene
Aim is to isolate the interesting, and ignore the boring
At the most simple level background subtraction is simply a
comparison between two image frames
At the more complex level many people have gotten phd’s for
better background learning techniques, and better differencing
algorithms
Brian Thorne Computer Vision in Python

49. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
In video security the camera mostly looks at the same boring
background
What we are usually interested in is when objects (eg people or
vehicles) enter or exit a scene
Aim is to isolate the interesting, and ignore the boring
At the most simple level background subtraction is simply a
comparison between two image frames
At the more complex level many people have gotten phd’s for
better background learning techniques, and better differencing
algorithms
Brian Thorne Computer Vision in Python

50. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
In video security the camera mostly looks at the same boring
background
What we are usually interested in is when objects (eg people or
vehicles) enter or exit a scene
Aim is to isolate the interesting, and ignore the boring
At the most simple level background subtraction is simply a
comparison between two image frames
At the more complex level many people have gotten phd’s for
better background learning techniques, and better differencing
algorithms
Brian Thorne Computer Vision in Python

51. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

52. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

53. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

54. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

55. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

56. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

57. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

58. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtraction
Simple Frame Differencing
1 To let the camera adjust to light levels, ignore the first few
frames.
2 Store a frame as the base frame.
3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel
(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.
3 Convert difference image to a one channel mask
4 Clean up small noise areas in the mask (with median filter,
erode, connected components)
5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python

59. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Background Subtract
Here I have placed a cellphone on my
cluttered desk
Can’t tell thats there is no green screen
Quick demo
Brian Thorne Computer Vision in Python

60. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

61. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Feature Point Detection
Feature point detection is implemented
in OpenCV you can do it in one call:
cvCornerHarris or cvGoodFeatures
To demonstrate the algorithm though -
we will go look at it in scipy.
Implementation derived from Jan
Solem
Brian Thorne Computer Vision in Python

62. Feature Detection
1 First convert to a grey scale
image
2 Showing the derivative in the x
and y directions
3 showing the millions of points of
interest
4 filtering them

63. Tools
Motivation & Background
Image Acquisition
Computer Vision in Python
Image Filtering
More Information
Background Subtraction
Summary
Feature Point Detection
Augmented Reality
Augmented reality is undergoing massive
growth
OpenCV provides the face detection
An AR game can easily be made in Pygame
using the webcam and face location as the
interface
Brian Thorne Computer Vision in Python

64. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

65. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Running on an OLPC
OLPC - provide children in developing
nations with access to knowledge, and
opportunities to "explore, experiment
and express themselves"
Includes Python and a webcam - thats
all you need for computer vision!
Here I am running OpenCV’s
facedetection on the XO laptop
Lots of Computer Vision work on the
XO has been done using pygame by
Nirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python

66. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Running on an OLPC
OLPC - provide children in developing
nations with access to knowledge, and
opportunities to "explore, experiment
and express themselves"
Includes Python and a webcam - thats
all you need for computer vision!
Here I am running OpenCV’s
facedetection on the XO laptop
Lots of Computer Vision work on the
XO has been done using pygame by
Nirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python

67. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Running on an OLPC
OLPC - provide children in developing
nations with access to knowledge, and
opportunities to "explore, experiment
and express themselves"
Includes Python and a webcam - thats
all you need for computer vision!
Here I am running OpenCV’s
facedetection on the XO laptop
Lots of Computer Vision work on the
XO has been done using pygame by
Nirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python

68. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Running on an OLPC
OLPC - provide children in developing
nations with access to knowledge, and
opportunities to "explore, experiment
and express themselves"
Includes Python and a webcam - thats
all you need for computer vision!
Here I am running OpenCV’s
facedetection on the XO laptop
Lots of Computer Vision work on the
XO has been done using pygame by
Nirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python

69. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Outline
1 Motivation & Background
What is Computer Vision?
Uses & Examples
2 Computer Vision in Python
Tools
Image Acquisition
Image Filtering
Background Subtraction
Feature Point Detection
3 More Information
Different platforms
Additional Tools
Brian Thorne Computer Vision in Python

70. IPython & MatPlotLib
Using IPython, an interactive shell can be used from deep
inside a nested loop in a running program.
In the code add
from IPython.Shell import IPShellEmbed
...
IPShellEmbed()()
Example
In [1]: from opencv import cv
In [2]: cv.cvAnd(diffImage,image, temp)
In [3]: timeit cv.cvAnd(diffImage,image, temp)
1000 loops, best of 3: 229 µs per loop
In [4]: from pylab import imshow, show
In [5]: imshow(temp)
Out[5]: <AxesImage object at 0x42489d0>
In [6]: show()

71. Motivation & Background
Computer Vision in Python Different platforms
More Information Additional Tools
Summary
Documentation & Support
The documentation in both SciPy and OpenCV was found to be
pretty good.... not entirely complete. The OpenCV book is really
good.
Remember Python is Free
Documentation is not going to be as extensive as for a professional
package like Matlab.... but you can help!
Support for these open source packages is almost entirely reliant on
experienced members of the community responding to requests on
message boards or mailing lists.
Brian Thorne Computer Vision in Python

72. Motivation & Background
Computer Vision in Python
More Information
Summary
Summary
For the scholar of computer vision research, using Python can
help in trying out new algorithms very quickly. The breadth of
the additional libraries available and the ease of integrating,
make new and novel solutions quickly realizable.
For someone just wanting to play around with some cool stuff,
its easy to dive in!
Limitations on using Python for CV
A major limitation of using Python would be when the
application is being developed for special embedded hardware
or when the best possible performance is required (at YOUR
expense)
Brian Thorne Computer Vision in Python

73. References
Thank You!
Thank you to
Raphaël Grasset - supervisor at HitLabNZ
Richard Green - computer vision lecturer
John Graves & Cristiano Soares for giving me detailed and
helpful feedback
Brian Thorne Computer Vision in Python

74. References
For Further Reading I
Library URL
Pygame http://pygame.org
OpenCV http://opencv.willowgarage.com
Numpy/Scipy http://scipy.org
Pycam http://pycam.googlecode.org
G. Bradski, A. Kaehler
Learning OpenCV.
O’Reilly Media, September 2008.
T. Oliphant
Guide to NumPy.
UT, Trelgol Publishing, 2006.
Brian Thorne Computer Vision in Python