2015 Artificial Intelligence Techniques at Engineering Seminar - Chapter 2 - Part 2: Genetic Algorithms
1. Artificial Intelligence Techniques
applied to Engineering
Part 2. Genetic Fuzzy Systems
Enrique Onieva Caracuel
@EnriqueOnieva
1.Fuzzy Logic
2.Genetic Algorithms
3.Genetic Fuzzy Systems
4.Applications to Intelligent
Transportation Systems: My Experience
2. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 2
Optimization
Is the process of looking for the best solution over
a set of feasible solutions
Applications:
Routes calculation
Process planning
Resource assignment
Pattern classification
Can be formulated as:
𝒎𝒊𝒏{𝒇(𝒙)│𝒙∈𝑿, 𝑿⊆𝑺 }
3. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 3
Example
Traveling Salesman Problem (TSP)
A set of nodes
To visit all the notes
One time in each node
Each arc (i-j) has a distance or cost associated
1
0 200 400 600 800 1000 1200 1400 1600 1800 2000
400
600
800
1000
1200
1400
1600
1800
2000
2200
2400
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
4. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 4
Example
Traveling Salesman Problem (TSP)
A set of nodes
To visit all the notes
One time in each node
Each arc (i-j) has a distance or cost associated
2
0 200 400 600 800 1000 1200 1400 1600 1800 2000
400
600
800
1000
1200
1400
1600
1800
2000
2200
2400
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
29! Feasible routes
(8,8418·1030
)
5. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 5
Example
Traveling Salesman Problem (TSP)
A set of nodes
To visit all the notes
One time in each node
Each arc (i-j) has a distance or cost associated
3
29! Feasible routes
(8,8418·1030
)
0 200 400 600 800 1000 1200 1400 1600 1800 2000
400
600
800
1000
1200
1400
1600
1800
2000
2200
2400
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
6. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 6
Representation
It has to be defined how genetic characteristics of the
individuals in the population are represented
Very important in the GA definition
It affects to the definition of genetic operators
(selection, crossover, mutation)
Types:
Bit string
Floating point
integer
LISP, Expressions, …
1
7. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 7
Representation
Requirements:
To allow to represent any solution
Not allowing to represent infeasible solutions
Adjusted to the problem
Small changes in the individual must represent small changes
in the solution
Easy to decode
2
8. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 8
Representation
1. Binary coding:
Selection problems
Back packing problems
2. Real coding:
Real optimization problems:
Find solution to:
𝑥1 + 10 · 𝑥2 + (𝑥3 · 𝑥4) + 𝑠𝑒𝑛(𝑥5 + 𝑥6) + cos(
𝑥7
𝑥8
) = 0
Distribution of 4 Gaussian
membership function in a fuzzy
system
3
0 2 4 6 8
0
0.5
1
9. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 9
Representation
3. Integer coding:
Grouping problems
Clustering
4. Permutation coding:
Sequencing problems:
Traveling salesman problem
Task to realize in a industrial chain
4
10. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 10
Evaluation
Evaluation (fitness):
Measures the quality of each individual
Allows to distinguish among good and bad individuals
Problem dependent
Fast execution (if possible)
Key: it decides the individuals to be selected
In general:
Is the most time consuming process in a real application
Can be a routine, a simulation or any external process
Approximate function can be used to reduce the execution
time
1
11. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 11
Genetic Algorithm 2
12. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 12
Genetic Algorithm
Generational Model Steady-State Model
13. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 13
Initialization
Uniform over the search space:
Can generate ANY individual
Binary string: [0,1] with equal probability
Real coding: uniform value in the interval
Integer coding: all the values have the same probability
Permutation coding: random permutation
Choose the initial population according with an
heuristic
14. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 14
Selection
Best individuals have higher chances of being
selected
Bad individuals must have any chance of being
selected
Selective Pressure: degree in which reproduction
is directed to best individuals
15. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 15
Selection Operators
Roulette: probability of being selected is
proportional to the fitness
𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 =
𝐹𝑖𝑡𝑛𝑒𝑠𝑠 𝑖
𝐹𝑖𝑡𝑛𝑒𝑠𝑠
Linear order: probability of
being selected is proportional to the order of the
individual
Tournament: K individuals are selected randomly,
the best of them is picked
Individual Fitness Probability
1 26 0,302
2 17 0,197
3 6 0,069
4 16 0,186
5 21 0,244
16. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 16
Crossover
Offspring has to inherit some characteristics from
each parent
It has to be designed according with the
representation
Must produce feasible individuals
17. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 17
Crossover Operators
One Point Crossover
N-Points Crossover
Uniform Crossover
1
Parents
Offspring
Offspring
Offspring
19. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 19
Crossover Operators
Permutation Coding
3
Parent 1 Parent 2
Child 1 Child 2
Order
20. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 20
Mutation
Must allow to reach any point at the search space
Variation size must be controlled
Must produce feasible solutions
Is applied with low probability over each individual
in the offspring obtained after the crossover
22. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 22
Replacement
The way in which individuals are replaced by new
offspring affects the selective pressure
Deterministic or randomized replacement
methods can be used
It can be decided that the best individual (or
individuals) are not replaced Elitism
1
23. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 23
Replacement
In a Steady-State Model, it can be replaced
The worst individual in the population
The most similar (from N) individual
The worst individual among a the set of the N most
similar
The most similar parent
2
24. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 24
Stopping Criteria
When the optimum is reached
Limited CPU resources:
Maximum number of generations
Limited amount of execution time
After a certain number of generations without
improvements found
25. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 25
Considerations
Execute more than one time
Use statistical measures (mean, deviation,…)
Easy to parallelize
Every search method needs equilibrium among:
Exploring the search space
Explode promising zones in the search space
Genetic Algorithms are general purpose search
methods. Genetic operators are used to maintain
this equilibrium
26. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 26
Considerations
Two opposite factors:
Convergence: to focus the search in promising regions
by selective pressure
Diversity: to avoid premature convergence
Selective Pressure: allow best individuals to be
selected to be crossed.
27. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 27
Considerations
Diversity
It is associated with differences between individuals in
the population
Low diversity: all the individuals are quite similar between
them
Low diversity premature convergence
Solutions:
To include diversity mechanisms in the process
To reinitialize once premature convergence is reached
28. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 28
Considerations
Diversity
With Mutation operator
Probability adaptation
To reduce it as the process run
Apply high probability to bad solutions and low probability to good
solutions
With the pairing for Crossover operator
Not cross individuals with themselves, their parents,
children,…
Incest prohibition to cross only if the are different enough
29. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 29
Considerations
Diversity
With the Crossover operator
Crossover operators with multiple parents
Crossover operators with multiple childs
30. Técnicas de Inteligencia Artificial aplicadas a la Ingeniería
@EnriqueOnieva 2014/2015 30
Extensions
Multimodal problems
Multiple solutions must be returned
Niching technique to converge to diverse local optima
Multi-objective problems
Multiple objectives must be satisfied
Mutually excluyent objectives
Quality – Prize
Power – Consumption
Accuracy - Complexity
There is not unique solution