1. Introduction to Machine
Learning
Lecture 24
Learning Classifier Systems
Albert Orriols i Puig
http://www.albertorriols.net
htt // lb t i l t
aorriols@salle.url.edu
Artificial Intelligence – Machine Learning
g g
Enginyeria i Arquitectura La Salle
Universitat Ramon Llull

2. Recap of Lecture 23
Michigan-style LCS
g y
Environment
Sensorial
Action Reward
state
Online rule evaluator:
• XCS: Q-Learning (Sutton & Barto, 1998)
Classifier 1
Learning
Any Representation:
y p Uses Widrow-Hoff delta rule
Classifier 2
Classifier
production rules,
genetic programs, System
Classifier n
perceptrons,
SVMs
Rule evolution:
Genetic Typically, a GA (Holland, 75;
Algorithm Goldberg, 89) applied on the
population.
Slide 2
Artificial Intelligence Machine Learning

3. Recap of Lecture 23
Main characteristics of XCS
Population-based method
Independent classifiers
Id d tl ifi
Works under a reinforcement learning paradigm, but can be
also applied t supervised l
l li d to i d learning ( d t f
i (and to function
ti
approximation)
Classifiers evolved b a genetic algorithm
Cl ifi l d by ti l ith
Slide 3
Artificial Intelligence Machine Learning

4. Today’s Agenda
Examples of domains
Another step toward cognitive systems
Anticipatory Classifier System
Slide 4
Artificial Intelligence Machine Learning

5. Applications of LCS
Examples of domains
p
Reinforcement learning
Supervised l
S i d learning
i
[Function approximation – not seen herein]
Real life
Real-life applications
Data Mining
Modeling market traders
Autonomous robotics
Modeling artificial ecosystems
…
Slide 5
Artificial Intelligence Machine Learning

6. Example in Reinf. Learning
Example: simple maze problem
p p p
XCS solves more complex reinforcement learning prob :
prob.:
Complex mazes
Mountain car
Slide 6
Artificial Intelligence Machine Learning

7. Example in Reinf. Learning
Performance in the Maze6 problem (Butz et al.)
p ( )
Slide 7
Artificial Intelligence Machine Learning

8. Example in Sup. Learning
Solving large, non-linear, complex boolean functions
000 0#######:0
000 1#######:1
001 #0######:0
001 #1######:1
010 ##0#####:0
010 ##1#####:1
011 ###0####:0
011 ###1####:1
100 ####0###:0
100 ####1###:1
101 #####0##:0
101 #####1##:1
110 ######0#:0
110 ######1#:1
111 #######0:0
111 #######1:1
Solving multiplexer problems up to 135 input variables
Slide 8
Artificial Intelligence Machine Learning

9. Current Real-Life Applications
Data mining
Most important application domain of LCSs
John H. Holmes
Epidemiologic study by means of LCSs
Hidden relationships among variables
p g
discovered by LCSs
Xavier Llorà et al.
Better than Human Capability in Diagnosing
Prostate Cancer Using Infrared Spectroscopic
imaging
gg
Many other applications and miners:
GALE, GAX GAssist, UCS,
GALE GAX, GAssist UCS MIPS …
See: Bull, Bernadó-Mansilla & Holmes (eds) Learning
Classifier Systems in Data mining. Springer (2008)
Artificial Intelligence Machine Learning Slide 9

10. Current Real-Life Applications
Modeling market traders
g
LETS project: Evolving artificial traders for successful market
trading (Sonia Sc u e bu g et a , 2007)
ad g (So a Schulenburg al, 00 )
Evolutionary economics:
Create trend followers
and value investors
Let them interact
Evolve a population of
strategies
Slide 10
Artificial Intelligence Machine Learning

11. Current Real-Life Applications
Autonomous Robotics
Robot shaping: Early efforts of Marco Dorigo and Marco Colombetti
(1997)
Small mobile robots equipped with sensors and motors
Robots connected in real time by various sorts of modem cable
Robots controlled by LCS, ICS, running on desktop computers
Constant stream of positive/negative rewards (bucket brigade)
Tasks solved:
Following lights
Gather food and run home
Hunt around for a light hidden behind and obstacle
Impressive results, high performance
Recent applications to model robotic problems performed in the
University of West England
Slide 11
Artificial Intelligence Machine Learning

12. Current Real-Life Applications
Modeling Artificial Ecosystems
g y
Eden: Artificial Life environment (Jon McCromak, 2004)
Model of an environment where evolvable rule based
rule-based
classifier systems drive agent behavior.
Autonomous LCSs or agents compete for limited
resources.
Agents can:
g
Make and listen to sounds
Forage for food
g
Encounter predators
Mate with each other
Goal: Maintain the audience in
tension without fitness needing the
audience explicitly perform fitness selection
Slide 12
Artificial Intelligence Machine Learning

13. Toward Cognitive Systems
Cognitive systems
g y
Cognitive systems are natural or artificial information
p ocess g systems, c ud g ose espo s b e o perception,
processing sys e s, including those responsible for pe cep o ,
learning, reasoning and decision-making and for
communication and action
LCS originally devised as cognitive systems
A step further
Anticipatory LCS
Slide 13
Artificial Intelligence Machine Learning

14. Anticipatory LCS
Anticipations influence cognitive systems
p g y
LCS learned:
Conditions x actions x prediction
Anticipatory learning processes learn
Condition x action x effect relations
Let’s see the Anticipatory LCS (ACS2)
Slide 14
Artificial Intelligence Machine Learning

15. ACS
ENVIRONMENT
A1
σt σt+1
Population [P]
Match Set [M]
C1 – A1 – E1
Action Set [A]
C2 – A2 – E2
C1 – A1 – E1
C3 – A3 – E3
C3 – A3 – E3
C4 – A1 – E4 C1 – A1 – E1
C4 – A1 – E4
C5 – A5 – E5 C4 – A1 – E4
C6 – A6 – E6
C6 – A6 – E6 C7 – A1 – E7
C7 – A1 – E7
C7 – A1 – E7 …
C9 – A9 – E9
C8 – A8 – E8
…
C9 – A9 – E9
…
Anticipatory Learning Process compare
Slide 15
Artificial Intelligence Machine Learning

16. Next Class
Big i t
Bi picture of what we have seen so far
f ht h f
New challenges in machine learning
Slide 16
Artificial Intelligence Machine Learning

17. Introduction to Machine
Learning
Lecture 24
Learning Classifier Systems
Albert Orriols i Puig
http://www.albertorriols.net
htt // lb t i l t
aorriols@salle.url.edu
Artificial Intelligence – Machine Learning
g g
Enginyeria i Arquitectura La Salle
Universitat Ramon Llull