ADVANCED OPTIMIZATION TECHNIQUES META-HEURISTIC ALGORITHMS FOR ENGINEERING APPLICATIONS - II
1. ADVANCED OPTIMIZATION TECHNIQUES
META-HEURISTIC ALGORITHMS FOR
ENGINEERING APPLICATIONS - II
Dr. M. K. MARICHELVAM,
ASSISTANT PROFESSOR,
DEPARTMENT OF MECHANICAL ENGINEERING,
MEPCO SCHLENK ENGINEERING COLLEGE,
SIVAKASI -626 005
E-mail: mkmarichelvamme@gmail.com
Mobile No: +91 9751043410
4. 29-Aug-16 4
Optimization
• Optimization is the act of obtaining the best result
under given circumstances.
• Optimization can also be defined as the process of
finding the conditions that give the maximum or
minimum of a function.
12. 12
Exact solution techniques
• An algorithm is described as a set of
instructions that will result in the solution to
a problem when followed correctly
• Unless otherwise stated, an algorithm is
assumed to give the optimal solution to an
optimization problem
– That is, not just a good solution, but the best
solution
13. 29-Aug-16 13
Exact solution Techniques
• Linear programming
• Integer programming
• Nonlinear programming
• Branch-and-bound algorithm, etc.
14. 14
Heuristic Algorithms
• A heuristic technique is any approach to
problem solving, learning, or discovery that
employs a practical method not guaranteed to
be optimal or perfect, but sufficient for the
immediate goals.
• Where finding an optimal solution is
impossible or impractical, heuristic methods
can be used to speed up the process of finding
a satisfactory solution.
• Heuristics can be mental shortcuts that ease the
cognitive load of making a decision.
16. Meta-Heuristics
A metaheuristic is formally defined as an iterative generation
process which guides a subordinate heuristic by combining
intelligently different concepts for exploring and exploiting
the search space, learning strategies are used to structure
information in order to find efficiently near-optimal solutions
[Osman and Laporte, 1996].
20. 29-Aug-16 20
FIREFLIES
• One of the family of insects
• Live in tropical environment
• Have wings
• Produce light chemically
• Yellow, green, pale-red lights
• Their larvae is called as glowworm
• ~2000 species.
21. 29-Aug-16 21
FIREFLIES BEHAVIOUR
– Their purpose of flashing:
Attarct mating partners (communication).
Attarct potential prey.
Protective warning mechanism.
– They have unique flashing pattern.
– In some species, females can mimic to hunt
other species.
– They have limited light intensity.
22. 29-Aug-16 22
FIREFLY ALGORITHM
• Firefly algorithm (FA) is a recently developed
nature-inspired meta-heuristic algorithm
developed by Dr. Xin-She Yang.
• The FA is inspired by the social behavior of
fireflies.
• Assumptions
– All fireflies are unisex.
– Attractiveness α Brigtness & Attractiveness α 1 /
Distance
– Brightness is determined by objective function.
23. 29-Aug-16 23
Three Idealized rules in FA
• All fireflies are unisex so that one firefly will be
attracted to other fireflies regardless of their sex.
• Attractiveness is proportional to their brightness,
thus for any two flashing fireflies, the less bright
one will move toward the brighter one. The
attractiveness is proportional to the brightness and
they both decrease as their distance increases. If
there is no brighter one than a particular firefly, it
will move randomly.
• The brightness of a firefly is affected or determined
by the landscape of the objective function. For a
maximization problem, the brightness may be
proportional to the objective function value. For the
minimization problem the brightness may be the
reciprocal of the objective function value.
24. 29-Aug-16 24
Pseudo code of the Firefly
algorithm
• Objective function f(x), x = (x1, ..., xd)T
• Generate initial population of fireflies xi (i = 1, 2,..., n)
• Light intensity Ii at xi is determined by f(xi)
• Define light absorption coefficient While (t <MaxGeneration)
• for i = 1 : n all n fireflies
• for j = 1 : i all n fireflies
• if (Ij > Ii ), Move firefly i towards j in d-dimension; end
• if
• Attractiveness varies with distance r via exp [−γr]
• Evaluate new solutions and update light intensity
• end for j
• end for i
• Rank the fireflies and find the current best
• end while
• Postprocess results and visualization
25. 29-Aug-16 25
Attractiveness of a Firefly
• The attractiveness of a firefly is determined
by its light intensity. The attractiveness may
be calculated by using the equation (1).
(1)
2
0
00 )( r
er
26. 29-Aug-16 26
Distance between Two Fireflies
• The distance between any two fireflies k
and l at Xk and Xl is the Cartesian distance
using the equation (2).
(2)
d
=k
ol,ok,lkkl )X(X=XX=r
1
2
27. 29-Aug-16 27
Movement of a Firefly
• The movement of a firefly k that is attracted
to another more attractive firefly l is
determined by the equation (3).
(3))α(rand+)X(Xeβ+X=X kl
2
klγr
okk
2
1
28. 29-Aug-16 28
Discrete Firefly Algorithm
The FA has been originally developed for
solving the continuous optimization
problems. The FA cannot be applied
directly to solve the discrete optimization
problems. In this work, the smallest position
value (SPV) rule is used to enable the
continuous FA to be applied to solve the
discrete HFS scheduling problems. For this,
a discrete firefly algorithm (DFA) is
proposed.
34. What is Scatter Search?
• Metaheuristic and Global Optimization
algorithm
• Use diversification (extrapolation) and
intensifications (interpolation) strategies,
not randomize
• Combining a set of diverse and high quality
candidate solutions by considering the
weights and constraints of each solution
• Introduced in 1970‟s, proposed by Fred
Glover in 1977
36. 36
Diversification Generation Method
• The idea behind the diversification generation
method is to generate a collection of diverse
solutions
• The quality of the solutions is not of importance
• Generation methods are often customized to
specific problems
• Can be totally deterministic or partially random
37. 37
Improvement Method
• Must be able to handle both feasible and
infeasible solutions
• It is possible to generate multiple instances of
the same solution
• This is the only component that is not
necessary to implement the scatter search
algorithm
38. 38
Subset Combination Method
• Construct subsets by building subsets of
Type 1, Type 2, Type 3 and Type 4 subsets
• For a RefSet of b there are approximately
(3b-7)*b/2 subset combinations
• The number of subsets can be reduced by
considering just one layer of subsets to
reduce computational time
39. 39
Solution Combination Method
• Generally problem specific, because it is
directly related to a solution representation
• Can generate more than one solution and
can depend on the quality of the solutions
being combined
• Can also generate infeasible solutions
• If a subset has been calculated on a previous
iteration it is not necessary to do the
calculation again
40. 40
Reference Update Method
• Objective is to generate a collection of both high
quality solutions and diverse solutions
• The number of Solution included in the RefSet is
usually less than 20
• Consists of the b1 best solutions from the
preceding step (solution combination or
diversification generation)
• Consists of the b2 solutions that have the largest
Euclidian distance from the current RefSet
solutions
• Multiple techniques are employed to update the
reference set
44. 2. Diversification Method
• By doing the same for the rest solutions,
then
Fitness Value
Solution 1 263
Solution 2 251
Solution 3 272
Solution 4 236
Solution 5 260
Solution 6 252
Solution 7 248
45. 3. Improvement Method
• Choose two worst solutions to
be improved
• Worst solution has biggest
fitness value
• 1st worst solution will be
improved by using NEH
algorithm
• 2nd worst solution will be
improved by using SPT
algorithm
Fitness Value
Solution 1 263
Solution 2 251
Solution 3 272
Solution 4 236
Solution 5 260
Solution 6 252
Solution 7 248
46. 3. Improvement Method (SPT)
• SPT algorithm use to improve fitness value for solution 1
• Calculate total time for each job, then order job ascending by total time
• Result of improvement
– New schedule for solution 1 is 3-5-4-1-2
– Makespan = 262
Machine 1 Machine 2 Machine 3 Machine 4 Machine 5 Total
Job 1 23 10 40 26 27 126
Job 2 30 18 30 39 37 154
Job 3 12 2 13 31 6 64
Job 4 50 4 8 15 41 118
Job 5 21 33 8 12 8 82
47. 3. Improvement Method (NEH)
• NEH algorithm use to improve fitness value for solution 3
• Calculate total time for each job
• Order job descending by the total time consume : 2 - 1 - 4 - 5 - 3
• Take top two job on the order (Job 2 and Job 1), then calculate the makespan for
each combination
– 1st combination : Job 1- Job 2 ; Makespan : 179
– 2nd Combination : Job 2 – Job 1 ; Makespan : 181
Machine 1 Machine 2 Machine 3 Machine 4 Machine 5 Total
Job 1 23 10 40 26 27 126
Job 2 30 18 30 39 37 154
Job 3 12 2 13 31 6 64
Job 4 50 4 8 15 41 118
Job 5 21 33 8 12 8 82
Take it to the next step
48. 3. Improvement Method (NEH)
• 1-2 combination is the most optimal combination
• Combine the 3rd job (job 4) with 1-2 combination
• Combinations : 1-2-4, 1-4-2, and 4-1-2
• Fitness value for each combination :
Combination Fitness
4-1-2 229
1-4-2 227
1-2-4 220 Take it to the next step
49. 3. Improvement Method (NEH)
• 4-1-2 combination is the most optimal combination
• Combine the 4th job (job 5) with 4-1-2 combination
• Combinations : 5-1-2-4, 1-5-2-4, 1-2-5-4, 1-2-4-5
• Fitness value for each combination :
Combination Fitness
5-1-2-4 251
1-5-2-4 242
1-2-5-4 228
1-2-4-5 228
Take it to the next step
50. 3. Improvement Method (NEH)
• 1-2-4-5 combination is the most optimal combination
• Combine the 5th job (job 3) with 4-1-2 combination
• Combinations : 3-1-2-4-5 , 1-3-2-4-5 , 1-2-3-4-5, 1-2-4-3-5,
1-2-4-5-3
• Fitness value for each combination :
Combination Fitness
3-1-2-4-5 240
1-3-2-4-5 255
1-2-3-4-5 237
1-2-4-3-5 234
1-2-4-5-3 234 NEH Result
51. New Reference Set
Job Order Fitness Value
Solution 1 3-5-4-1-2 262
Solution 2 3-4-2-1-5 251
Solution 3 1-2-4-5-3 234
Solution 4 2-1-4-3-5 236
Solution 5 1-5-4-3-2 260
Solution 6 5-2-1-4-3 252
Solution 7 1-5-2-4-3 248
52. 4. Subset Generation
Job Order Fitness Value
Solution 1 3-5-4-1-2 262
Solution 2 3-4-2-1-5 251
Solution 3 1-2-4-5-3 234
Solution 4 2-1-4-3-5 236
Solution 5 1-5-4-3-2 260
Solution 6 5-2-1-4-3 252
Solution 7 1-5-2-4-3 248
• Reference set R = 3; Ra=2 (best solution); Rb=1 (worst solution)
– R-1 = 2 max type subset
Ra
Rb
53. 4. Subset Generation
(Type 1)
• a1 = solution 3 ; a2 = solution 4 ; b = solution 1
• Each subset type-1 has 2 values
• Every subset is a set combination of a1, a2, and b
• Subset type 1 :
(a1,a2) , (a1,b), (a2,b) = (3,4), (3,1), (4,1)
• Find new combination solution by using
neighborhoods method
– Find diverse value between job order one each subset
combination solutions
54. 4. Subset Generation
(Type 1)
• Find new solutions by exchanging jobs between solution 3 and
solution 4
1 2 4 5 3
Solution 3
2 1 4 3 5
Solution 4
1 4 5 3
New Solution 1
2
2 4 3 5
New Solution 2
1
56. 4. Subset Generation
(Type 1)
• Do neighborhood method for the rest member
of subset type 1
– Exchanging jobs between solution 3 and solution 1
– Exchanging jobs between solution 4 and solution 1
1 2 4 5 3
Solution 3
2 1 4 5 3
Solution 4
2 4 3 5
New Solution 5
1
1 4 5 3
New Solution 6
2
57. 4. Subset Generation
(Type 1)
Neighborhood New solution Combination Fitness
Solution 3 and Solution 4
New Solution 1 2-1-4-5-3 236
New Solution 2 1-2-4-3-5 234
New Solution 3 2-1-4-5-3 236
New Solution 4 1-2-4-3-5 234
New Solution 5 1-2-4-3-5 234
New Solution 6 2-1-4-5-3 236
New Solution 7 1-2-4-3-5 234
New Solution 8 2-1-4-5-3 236
Solution 3 and Solution 1
New Solution 9 1-5-4-3-2 260
New Solution 10 3-2-4-5-1 249
New Solution 11 3-2-4-1-5 242
New Solution 12 1-5-4-2-3 254
New Solution 13 3-1-4-5-2 263
New Solution 14 5-2-4-1-3 252
New Solution 15 2-5-4-1-3 237
New Solution 16 1-3-4-5-2 263
Solution 1 and Solution 4
New Solution 17 2-5-4-1-3 237
New Solution 18 3-1-4-2-5 247
New Solution 19 3-1-4-5-2 263
New Solution 20 2-5-4-3-1 239
New Solution 21 1-5-4-3-2 260
New Solution 22 2-3-4-1-5 239
New Solution 23 3-2-4-1-5 242
New Solution 24 5-1-4-3-2 260
58. 4. Subset Generation
(Type 2)
• Each subset type-2 has 3 values
• Every subset is a set combination of a1, a2, and b
• Subset type 1 : (a2,b) a1 = (4,1) 3
• The most optimal combination of neighborhood (4,1) is New Solution
17
• Find new combination solutions by using neighborhood method on
New Solution 17 and Solution 3
1 2 4 5 3
New Solution 17
2 1 4 3 5
Solution 3
1 4 5 3
New Solution 25
2
2 4 3 5
New Solution 26
1
59. 4. Subset Generation
(Type 2)
New solution Combination Fitness
New Solution 25 1-5-4-2-3 254
New Solution 26 2-1-4-5-3 236
New Solution 27 5-2-4-1-3 252
New Solution 28 1-5-4-2-3 254
New Solution 29 2-1-4-5-3 236
New Solution 30 5-2-4-1-3 252
• The most optimal combination of 37 solution (7 seed + 30 new) is
Solution 3 (1-2-4-5-3), with Fitness value = 234
61. 29-Aug-16 61
Cuckoo Search (CS) algorithm
• Cuckoo search (CS) algorithm is a new nature-
inspired meta-heuristic algorithm developed by
Yang & Deb (2009).
• CS algorithm was inspired by the obligate
brood parasitic behavior of some cuckoo
species in combination with the Lévy flight
behavior of some birds and fruit flies in nature.
• The breeding behaviour and the Lévy flights
will be discussed in the following sections.
62. 29-Aug-16 62
Cuckoo Breeding Behaviour
• Some of the cuckoo species lay their eggs in the
nests of other host birds.
• The cuckoos often select the recently spawned
nests instinctly.
• They may remove others eggs to increase the
hatching probability of their own eggs.
• Some host birds can engage direct conflict with
the intruding cuckoos.
• If a host bird discovers the eggs are not their
owns, they will either throw these alien eggs away
or simply abandon its nest and build a new nest
elsewhere.
63. 29-Aug-16 63
Cuckoo Breeding Behaviour
• Some cuckoos have evolved in such a way that female parasitic
cuckoos are often very specialized in the mimicry in colour and pattern
of the eggs of a few chosen host birds.
• This will reduce the probability of their eggs being abandoned. This
also increases their reproductivity.
• Furthermore, the timing of egg-laying of some cuckoos is also
amazing.
• The cuckoos often choose a nest where the host bird just laid its own
eggs.
• In general, the cuckoo eggs hatch slightly earlier than their host eggs.
• Once the first cuckoo chick is hatched, the first instinct action it will
take is to evict the host eggs by blindly propelling the eggs out of the
nest.
• This will increase the cuckoo chick‟s share of food provided by its host
bird.
• Moreover, a cuckoo chick can also mimic the call of host chicks to
gain access to more feeding opportunity.
64. 29-Aug-16 64
Rules of CS algorithm
• Each cuckoo lays one egg (solution) at a time, and
dumps its egg in a randomly chosen nest. That is, an
egg represents a solution. As there is one egg in one
nest, then it can be assumed that an egg is equivalent
to a nest and a solution.
• The best nests with high quality eggs/solutions will
carry out to the next generation. Here is the best
means the solution with the best (minimum)
objective values (for minimization problems).
• The egg laid by a cuckoo can be discovered by the
host bird with a probability Pa and a nest will then
be built. That is to say, a fraction Pa of the n nests
being replaced by new nests (with new random
solutions at new locations).
65. 29-Aug-16 65
Pseudo code of the cuckoo search
algorithm
• Start
• Objective function f(x), x = (x1, x2, ..., xd)
• Generate initial population of n host nests xi (i = 1, 2,..., n)
• While (t <MaxGeneration)
• Get a cuckoo randomly (say a) by Lévy flights
• evaluate its quality/fitness Fa [proportional to f(x)]
• Choose a nest among n (say b) randomly
• if (Fa is better than Fb),
• replace solution b by the new solution;
• end
• A fraction (Pa) of worse nests is abandoned and new ones are built;
• Keep the best solutions (or nests with quality solutions);
• Rank the solutions and find the current best
• end while
• Post process results and visualization
• End
66. 29-Aug-16 66
Parameters Used in CS algorithm
Sl. No. Factors Levels
1 Probability of worse
nests to be abandoned Pa
0.01 (low)
0.10 (medium)
0.50 (high)
2 Step size α1 0.05 (low)
0.50 (medium)
1.0 (high)
3 Random step length λ0 1.50 (low)
2.00 (medium)
3.00 (high)
67. Application of the CS Algorithm
•Engineering optimization problems
•NP hard combinatorial optimization problems
•Data fusion in wireless sensor networks
•Nanoelectronic technology based operation-amplifier
(OP-AMP)
•Train neural network
•Manufacturing scheduling
•Nurse scheduling problem
68. 29-Aug-16 68
HYBRID ALGORITHMS
• NEED FOR HYBRIDIZARTION
• TYPES
1. PARALLEL HYBRIDIZATION
2. SERIAL HYBRIDIZATION
• EXAMPLES
1. GENETIC SCATTER SEARCH
2. PSO – GA
3. GA – PSO
4. HYBRID FIREFLY
5. HYBRID CUCKOO SEARCH, etc.
70. 29-Aug-16 70
REFERENCES
• Dorigo, M. Optimization, learning and natural algorithms
(in italian). Ph.D. thesis, DEI, Politecnico di Milano, Italy.
pp. 140, 1992.
• Glover, F. (1977) „Heuristics for integer programming
using surrogate constraints‟, Decision Sciences, Vol. 8,
No. 1, pp.156–166.
• Holland, J. H. Adaption in natural and artificial systems.
The University of Michigan Press, Ann Harbor, MI. 1975.
• Osman, I.H., and Laporte, G. “Metaheuristics: A
bibliography”. Ann. Oper. Res. 63, 513–623, 1996.
71. 29-Aug-16 71
REFERENCES
• Kennedy, J. and Eberhart, R. “Particle Swarm
Optimization”, Proceedings of the 1995 IEEE International
Conference on Neural Networks, pp. 1942-1948, IEEE
Press, 1995.
• Kirkpatrick, S., Gelatt. C. D., and Vecchi, M. P.
“Optimization by simulated annealing”, Science, 13 May
1983 220, 4598, 671–680, 1983.
• Marichelvam, M. K. (2012). An improved hybrid Cuckoo
Search (IHCS) metaheuristics algorithm for permutation
flow shop scheduling problems. International Journal of
Bio-Inspired Computation, 4(4), 200-205.
72. 29-Aug-16 72
REFERENCES
• Marichelvam, MK & Prabaharan, T 2012, „A bat algorithm
for realistic hybrid flow shop scheduling problems to
minimize makespan and mean flow time‟, ICTACT
Journal on Soft Computing, vol. 3, no. 1, pp. 428-433
• Marichelvam, MK, Prabaharan, T, Yang, XS & Geetha, M
2013, „Solving Hybrid Flow Shop Scheduling Problems
using Bat Algorithm‟, International Journal of Logistics
Economics and Globalization, vol. 5, no. 1, pp. 15-29.
• Marichelvam, MK, Prabaharan, T & Yang, XS 2014, „A
Discrete Firefly Algorithm for the Multi-Objective Hybrid
Flowshop Scheduling Problems‟, IEEE Transactions on
Evolutionary Computation, vol. 18, no. 2, pp. 301-305.
73. 29-Aug-16 73
REFERENCES
• Marichelvam, M. K., & Geetha, M. A hybrid discrete
firefly algorithm to solve flow shop scheduling problems
to minimize total flow time. International Journal of Bio-
Inspired Computation. In press.
• Marichelvam, M. K., & Geetha, M. A hybrid cuckoo
search metaheuristic algorithm for solving single machine
total weighted tardiness scheduling problems with
sequence dependent setup times. International Journal of
Computational Complexity and Intelligent Algorithms. In
press.
• Omur Tosun & Marichelvam, M. K. Hybrid bat algorithm
for flow shop scheduling problems. International Journal
of Mathematics in Operational Research. In press.
74. 29-Aug-16 74
REFERENCES
• C. Subramanian, K. Subramanian and A. S. S. Sekar, “A
New Parameter Free Meta-Heuristic Algorithm for
Continuous Optimization: African Wild Dog Algorithm”,
European Journal of Scientific Research, Vol. 92, No. 3,
pp. 348 – 356, 2012.
• Yang, X.S. (2008). Nature-Inspired Metaheuristic
Algorithms, Luniver Press, Frome, UK.
• Yang, X.S. and Deb, S. (2009) „Engineering optimisation
by Cuckoo Search‟, International Journal of Mathematical
Modelling and Numerical Optimisation, Vol. 1, No. 4,
pp.330–343.
75. 29-Aug-16 75
REFERENCES
• Yang, X.S. (2010) „A new metaheuristic bat-inspired
algorithm‟, in Gonzalez, J.R. et al. (Eds.): Nature Inspired
Cooperative Strategies for Optimization, Vol. 284, pp.65–74,
(NICSO 2010)Springer, SCI.
• R. Zhao and W. Tang, “Monkey Algorithm for Global
Numerical Optimization”, Journal of Uncertain Systems, vol.
2, no. 3, pp. 165-176, 2008.
• http://www.egyptscience.net
• http://www.slideshare.net/HarshadaGurav/bat-
algorithmbasics
• http://www.slideshare.net/hasangok/firefly-algorithm-
19000019
76. 29-Aug-16 76
ACKNOWLEDGEMENTS
• MY FANILY MEMBERS
• THE PRINCIPAL AND MANAGEMENT OF MEPCO
SCHLENK ENGINEERING COLLEGE, SIVAKASI
• THE PRINCIPAL AND MANAGEMENT OF KAMARAJ
COLLEGE OF ENGINEERING & TECH., VIRUDHUNAGAR
• Dr. P. NAGARAJ Sr. PROFESSOR & HOD, MECHANICAL
ENGINEERING, MEPCO SCHLENK ENGINEERING
COLLEGE, SIVAKASI
• MY SUPERVISOR Dr. T. PRABAHARAN, PROFESSOR,
MECHANICAL ENGINEERING, MEPCO SCHLENK
ENGINEERING COLLEGE, SIVAKASI
77. 29-Aug-16 77
ACKNOWLEDGEMENTS
MY DOCTORAL COMMITTEE MEMBERS
1. Dr. N. JAWAHAR, DEAN,
THIAGARAJAR COLLEGE OF ENGINEERING,
MADURAI.
2. Dr. S. SARAVANASANKAR,
VICE CHANCELLOR,
KALASALINGAM UNIVERSITY,
SRIVILLIPUTTUR.
3. Dr. S. KRISHNAIAH, PROFESSOR,
ANNA UNIVERSITY,
CHENNAI.
4.Dr.S. P. NATCHIAPPAN
THIAGARAJAR COLLEGE OF ENGINEERING,
MADURAI.
78. 29-Aug-16 78
ACKNOWLEDGEMENTS
MY CO-AUTHORS
1. Xin-She Yang
School of Science and Technology, Middlesex
University, London NW4 4BT, U.K.
2. M. Geetha
Department of Mathematics,
Kamaraj College of Engineering and Technology,
Virudhunagar, Tamilnadu, 626 001
79. 29-Aug-16 79
ACKNOWLEDGEMENTS
MY CO-AUTHORS
3. Ömür Tosun
Department of International Trade and Logistics,
Ayşe Sak School of Applied Sciences,
Akdeniz University, Yeşilbayır, Antalya, Turkiye
4. A. Azhagurajan
Department of Mechanical Engineering,
Mepco Schlenk Engineering College,
Sivakasi – 626 005.